Kamper brings up the point that we can't just assume the dimension of time behaves the same way as the three spatial dimensions. Why not? Let's think about it for a minute. I can walk to the corner store from my house, and then I can walk back to my house from the corner store. Or I can run. I can slowly climb up a ladder, sit up there for a minute, and then slide down a twisty slide. Point being, in all of the first three dimensions, the direction and speed of motion is not limited - more or less.
But it would at least seem like time is different in this regard. If we think again about time as a line, we realize that we can only move along that line in one direction and only at one speed. The idea of speeding up and slowing down time or going "backward" in time doesn't make sense. Right? Wrong. Or, kind of wrong. The "pace" of time is effected by gravity. Time runs slower in areas closer to a massive object and and faster in areas further from a massive object. The difference can be extremely small, but it is enough that GPS satellites in Earth orbit have to have their on-board clocks routinely resynched with clocks on Earth's surface. Theoretically, time stops altogether at the center of a black hole, not a small difference at all.
More to the point, it is obvious at least that we humans are not capable of traversing time with as much freedom as we can space, but that doesn't necessarily say anything about the nature of time as much as it can say something about the nature of humans. Human beings are not capable of flying, but we developed machines that were; and we got inside them. That's the story of our species. We encounter a boundary that we can't cross and instead of going somewhere else, we develop a technology that crosses that boundary for us. That said, we are finite creatures with finite abilities and so it may be that time travel is possible, just not for us - or our machines.