2

Does there exist an algorithm for which an exact complexity provably cannot be expressed in closed-form?

Here closed-form means a finite composition of addition, subtraction, product, division, factorial, power with any exponent, logarithm, trigonometric function, inverse trigonometric function, hyperbolic function, and inverse hyperbolic function. You may choose a subset of the above functions to allow in a closed-form expression; this makes the problem easier. However, the larger the set of allowed functions, the better, since this also answers the problem for the subsets.

Exact complexity is a function from the input-set to a real number. You may group the input by some property, and then study exact worst-case complexity instead (or exact best-case complexity).

Any computational model will do, as well as counting any resource (e.g. number of comparisons). To close off a trivial solution, a function without a closed-form expression cannot be a primitive operation of the computational model.

If yes, is there a simple example of such an algorithm?

kaba
  • 411
  • 3
  • 8
  • In which sense do you want to understand "cannot"? I don't that there are any functions that provably have no closed form, but there are many for which we don't know them. Cost functions only known in implicit form often arise in algorithm analysis, they are nothing special. In fact, they are rather the norm once you leave textbook-land. – Raphael Jul 17 '16 at 12:09
  • @Raphael Suppose you have analyzed an algorithm to have a complexity expressed by a simple geometric series. This is not closed-form, because it uses an infinite number of operations. However, you can use the formula for geometric series to bring it into a finite combination of subtraction and division, which is closed-form. There are many expressions for a function; it is sufficient for one of them to be closed-form. I want provably no closed-form. – kaba Jul 17 '16 at 12:39
  • I see. Then you should probably say "can be computed using $O(1)$ arithmetic operations" and not "commonly named functions"; the latter does not preclude me calling anything by a fancy name that looks short but only hides "infinitely many operations". Arguably, "$n!$" is such a shorthand, and even $\log$ and $\sqrt{}$. – Raphael Jul 17 '16 at 12:41
  • @Raphael Factorial, square root, and logarithm are fine too. The point is to use functions which are of general use, and not tailored to a very narrow use. A good test could be whether Matlab, or any other programming language, provides an implementation to approximate it in its standard library. – kaba Jul 17 '16 at 12:46
  • Which would include many functions that are only "closed" by being aliases of other stuff. I maintain that you have only a vague idea of what you want, and thus your question is essentially not answerable. – Raphael Jul 17 '16 at 14:35
  • @Raphael I made the question more specific now. I wish to give freedom to choose the set of allowed functions yourself, to get at least some answers. The problem gets harder the more there are allowed functions. The functions are in the sense of mathematics; there are no aliases. – kaba Jul 17 '16 at 15:29

4 Answers4

4

The runtime of an algorithm that computes the Ackermann function can't be expressed using primitive recursive functions. All commonly known named functions (well, besides the Ackermann function) are primitive recursive as far as I know.

adrianN
  • 5,951
  • 18
  • 27
  • This seems like a good direction.Especially if we make the restriction that the complexity function and the functions in the closed-form expression are both non-negative-integer-valued. – kaba Jul 17 '16 at 01:12
4

Here's a trivial example of an algorithm whose running time has no closed form:

let x = f(n)
while x >= 0: x := x - 1

where n is the parameter and f is a function with no closed form and with a running time that has a closed form.

It's always possible to construct an algorithm with any (computable) running time. There are as many ways as you like to do nothing. Sure, this isn't the optimal way to do nothing, but we don't limit the study of algorithms to optimal algorithms. In fact, we need to evaluate the complexity of an algorithm before we can say whether it's optimal.

Gilles 'SO- stop being evil'
  • 43,613
  • 8
  • 118
  • 182
  • This example is indeed valid if the computational model allows, as a primitive operation, to evaluate in unit time at least some function f without a closed-form expression; I did say any computational model in the initial post. However, I would like to close off this possibility as a trivial solution, forcing the algorithm f to be further specified. Is that ok? – kaba Jul 17 '16 at 00:19
  • @Kaba What do you mean by “forcing the algorithm f to be further specified”? f isn't an algorithm, it's a function, and I don't understand what you mean by “further specified”. – Gilles 'SO- stop being evil' Jul 17 '16 at 00:26
  • The f is either an algorithm, or a primitive operation of the computational model. If I reject f being a primitive operation, then f has to be computed by an algorithm using simpler primitive operations. That algorithm f would then have to be specified. – kaba Jul 17 '16 at 00:31
  • @Kaba The answer is generic; pick any computable $f$ which does not have a "closed form". It's easy to adapt the algorithm to dominate the cost of computing $f$, whatever that may be. – Raphael Jul 17 '16 at 10:38
  • @Raphael I understand. However, I want the algorithm to be specified as a sequence of primitive operations of the computational model. Sub-algorithms can be used provided they are specified. For example, an executable Python program would do. – kaba Jul 17 '16 at 11:22
  • @Kaba You should edit your question to reflect that. You are currently asking whether such algorithms exist. – Raphael Jul 17 '16 at 12:07
  • @Raphael Existence is also fine; there are two questions. The problem is that I can't "pick"; there might not exist an algorithm whose complexity cannot be expressed in closed-form. When f is not a primitive operation, this answer reduces to: "Assume f exists. Then f exists." – kaba Jul 17 '16 at 12:29
0

You can always find the exact number of operations by simulating the algorithm and counting all operations. If you accept that as "closed form" then obviously you can give the exact number of operations for any algorithm.

If not, then I assume that you would want to know if it is possible, after careful human analysis of the algorithm, write a program that looks at any input for the algorithm, and within reasonable time and without simulating the algorithm calculates a reasonable approximation for the running time of the algorithm for that input.

Take a simple algorithm like Quicksort. You choose a pivot, and the choice of the pivot, good or bad, massively affects the runtime. It is possible that some input data always produces a really bad choice of the pivot, and excessive runtime. Even if you introduce a pseudo-random generator to pick the pivot, there's still the possibility of excessive runtime. To give a decent approximation for the runtime for some input, you don't really have any choice but simulating the algorithm.

Well, there's probably someone more clever than me, and maybe it is possible to analyse Quicksort, but then Quicksort is a very simple algorithm.

PS. The OP has now stated that he wants the running time of an algorithm based on some simple parameter. That's nonsense. He's not going to get that. Again the example Quicksort: The runtime depends on the exact ordering of the values in the array in a highly complex way. For any implementation, I can produce an array that maximises the running time, and a single change of one item in that array total changes it.

gnasher729
  • 29,996
  • 34
  • 54
  • 1
    "You can always find the exact number of operations by simulating the algorithm and counting all operations" -- Only for finitely many inputs, which prevents you from learning much about cost functions. – Raphael Jul 16 '16 at 21:14
  • The complexity function exists as a function; I take it as given, as a mathematician would:) Then the problem is to develop a sufficiently tricky algorithm for the complexity to avoid closed-form expressions. – kaba Jul 17 '16 at 00:02
  • @Raphael: So what? You can only evaluate a function for finitely many arguments. – gnasher729 Jul 17 '16 at 11:58
  • @gnasher729 The OP clearly wants a formula in $n$ (or some other parameter), otherwise the question is rather meaningless. – Raphael Jul 17 '16 at 12:08
  • @Raphael That's correct. – kaba Jul 17 '16 at 12:32
  • @gnasher729 Intuitively you have the right idea, in that even an easy algorithm gets so complex as to not be expressible in closed-form. What is missing is a proof of that. I was being unclear with "That's correct.". For a closed-form exact complexity, there is some parameter, such as the input-element, which is being mapped by a closed-form expression to a real number. In the case of exact worst-case analysis, that parameter may be an integer, or several integers. – kaba Jul 18 '16 at 10:00
  • What do you mean by "input-element"? And what parameters do you suggest for Quicksort? – gnasher729 Jul 18 '16 at 23:01
  • @gnasher For quicksort, the input set is $\mathbb{N}^$, the set of sequences of natural numbers (say). The exact complexity is a function $f : \mathbb{N}^ \to \mathbb{R}$. An example of a closed-form expression: $f(x) = \sqrt{|x|}$, where $|x|$ is the length of the sequence $x$. If I group the input by sequence length, then I could consider the exact worst-case complexity $g : \mathbb{N} \to \mathbb{R}$. An example of a closed-form expression: $g(n) = n^2$. – kaba Jul 19 '16 at 09:44
  • Correction: The example for $f$ is not valid for a closed-form expression if sequence length is not allowed in a closed-form expression. It is merely an example of an expression for an exact complexity. – kaba Jul 19 '16 at 10:06
0

There are plenty of simple algorithms whose running-time is unknown in any form. Therefore, their "complexity" can not be expressed in closed-form either, for suitable notions of "closed-form".

Note that, ironically, some unknown running-time functions may have closed forms in the rather vague sense of "using commonly used names for things" since some famous unknown quantities have been given names.

Community
  • 1
Raphael
  • 72,336
  • 29
  • 179
  • 389
  • Not knowing the complexity of an algorithm now (say, the Collatz example in the link) does not exclude it from having a closed-form expression (in terms of functions often used in mathematics). The complexity exists as a function, and that either has a closed-form or not. I hope I did not misunderstood! – kaba Jul 16 '16 at 23:52
  • @Kaba Personally, I think "unknown" precludes "closed form", but that may be a matter of perspective. (Goes to show the vagueness of the question...) – Raphael Jul 17 '16 at 10:35
  • The complexity of every new algorithm begins in the "unknown" state. If I later sit down to analyze the exact complexity as n^2 + 3, then I have shown it to have a closed-form expression. However, the algorithm had this property even before I analyzed it. Again, I hope I have understood your sense of "unknown" correctly:) – kaba Jul 17 '16 at 11:10
  • @Kaba I mean "unknown" in the sense of "nobody knows, and we do not expect anyone to find out soon, and not for a lack of trying"; much like it is unknown whether P=NP even though it certainly either is or not. In the sense of the other question I link, that is. – Raphael Jul 17 '16 at 12:06
  • Ok, then I understood you correctly:) – kaba Jul 17 '16 at 12:33