Why call it 'Time Complexity'?

Question

P.S. I have added the tag 'history', if there is any historical connotation.

Also, I found this question What is running time of an algorithm? but I am not satisfied with answers.

Answer 1

Perhaps the earliest place in which time complexity appears is On the computational complexity of algorithms by Hartmanis and Stearns. Their goal is to study computation complexity, which they define as follows:

The computational complexity of a sequence is to be measured by how fast a multitape Turing machine can print out the terms of the sequence.

Their first section, in which they prove (among else) a time-hierarchy theorem, is about "time-limited computations". They explicitly mention their concept of time:

The machine operation is our basic unit of time.

The reference is to a multitape Turing machine, which they diligently define.

The intention here is to model the running time of algorithms using abstract machines, using number of steps as a proxy for time. Anticipating your criticism, they mention:

Furthermore, the [complexity] classes are independent of time scale or of the speed of the components from which the machines could be built, as there is a "speed-up" theorem which states that $S_T = S_{kT}$ [i.e., $\mathsf{TIME}(T(n))$ = $\mathsf{TIME}(kT(n))$] for positive numbers $k$.

That is, multitape Turing machines can always be sped up by an arbitrary constant, and so there is no harm in associating number of steps with running time, since time complexity classes are "scale free".

Answer 2

I think, though I don't have any references to back this up, that it's just a convenient name that has a ring of truth to it.

If you imagine implementing a standard Turing machine, it does seem reasonable that every step of your actual, physical machine will take the same amount of time. So, for a Turing machine, time and number of steps are the same thing, up to a constant factor. That isn't true for more complicated machines – for example, a single step of a RAM could involve moving the tape heads an arbitrary distance – but the analogy is good enough.

All names seem to have disadvantages. "Time complexity" sounds like it's measured in seconds. Something like "step complexity", "operation complexity" or "instruction complexity" might be misunderstood as referring to the complexity of the individual steps, rather than to complexity measured by the number of steps. But these aren't big disadvantages: it only takes a moment to explain that "time" doesn't literally mean time or that "step complexity" doesn't mean the complexity of the steps.

If we were starting again from scratch, I think either "time complexity" or "step complexity" would be a reasonable name. I can't think of any other term that's both reasonably short and more accurately conveys the concept of complexity measured in terms of the number of computational steps.

_{And be thankful that we didn't call it "type 1" complexity. I'm looking at you, statistics and diabetes.}

Answer 3

Time complexity is a formal model (an abstraction) of program running time. Although on the face of it you are right that it really measures the number of steps, it is asymptotically no different from the actual running time of the machine (Turing machine or any other model of computation). Therefore I disagree that there is any problem with the terminology.

Think about it from the programmer's perspective. When you write a piece of code, say

 for i = 1 ... n :
     for j = 1 ... i :
          print j
     print newline

you can't (as a programmer) actually predict how long the program will take to run in seconds, with accuracy. Moreover the number of seconds depends on the exact platform on which you run the code, level of parallelization, what file or output you are printing to, etc.

But what you can measure is the number of steps your code runs -- that is, its time complexity, as a function of $n$. You simply count the number of times a print statement is executed. This is -- up to a constant -- a good and correct estimate of the actual time the program will take to run, in seconds.

In summary, the concept of time complexity is exactly the same concept that programmers use to think about their code's performance, and the abstraction is the same as the actual running time up to a constant.

Answer 4

From what I can tell, your appeal to the terms "step complexity" and "operation complexity" are a way of attaining a more objective definition of an alogithm's behavior, without tying it down to the actual execution time of the algorithm on an arbitrary machine.

However, "step" and "operation" are not so objective, either. Different CPUs can have vastly different instruction sets. For example, in the x86 instruction set, MOV EAX, [EDX + EBX*4 + 8] is a single instruction! That's despite it doing 1 multiplication, 3 additions, and 1 move.

Perhaps you could say "okay, well then we'll count it as 5 operations." Well that gets really complicated, too. These could very well be done in one shot by hardware, or perhaps as a decomposition of simpler instructions as orchestrated by the processor's microcode. Not only is it not defined which of these two approaches a processor uses, but the notion of these "simpler instructions" is also blurred. Who's to say multiplication is a single instruction? It could be implemented as a sequence of additions, for all you know (it's not, don't worry, but the point stands).

Anyhow, talking about "time complexity" makes sense, because we don't really care about steps or operations. What we really care about is how steps or operations can be used as a proxy for time. E.g. we don't care if one algorithm takes 10 steps or another takes 20, if the first algorithm takes 10 seconds and the other takes 2 seconds.

Answer 5

In practice an actual machine does not spend a particular time for each operation that is equal to any other machines length of time. Hence what we measure is a general trend of growth.

Consider addition on a machine nowadays.

Consider addition on a mechanical computer.

It is not unreasonable to use both as models to measure the length of time an algorithm takes to complete. However, the issue is that both have wildly different EXACT amounts. All we can reason is how changing the input changes the runtime of the algorithm.

A better way to consider is that runtime is a function of the length of time of each step you assume to take one unit of time. Then, time complexity is a result of analyzing that function. In particular because those times will vary from actual machine to actual machine.

I.E. give the abstract machine abstract times to complete each immutable step so that it can apply to any such machine implemented in the real world.