Reasoning on Efficiency (2)

Question

Two algorithms to solve a particular problem can have theur efficiency compared using the $O$ and $o$ notation. However, this is very crude method, and tells us no information on how more effective one is than the other.

Is there a yard stick that can be applied to ALL algorithms, with ALL time complexities, that can be used to evaluate the efficiency of two algorithms?

I actually developed a method for this, but it applies only to polynomial algorithms.

MY METHOD.

Suppose we want to compare two algorithms for accomplishing a task $f_i(n), g_i(n)$. $f_i(n) = o\left(g_i(n)\right)$
A simple way to compare $f_i$ and $g_i$, is to find their ratio $r_i(n)$
$$r_i(n) = \frac{g_i(n)}{f_i(n))}$$
Express $r_i(n)$ as a polynomial of ,$n$.
$$r_i(n) = n^{k}, \{k: k \in \Bbb R\}$$
$$m = \lceil{k}\rceil$$

$R_i(n) = m^{th}$ derivative of $r_i(n)$.
It follows that $R(n)$ is a constant.

My method works nicely for polynomial algorithms, but is completely useless for other time complexities. Is there a more effective yardstick? One that applies to all complexities?

Please don't use spoiler to show a block of text with LaTeX in the question because it gets unreadable. The Landau notation is used for asymptotic behaviour. From the mere counting of "ALL" in your question the answer will be no. Also you are trying to compare efficiency, this requires additional considerations like cache, architecture etc. If you manage to show some analysis that suit your needs for polynomials, a fine grained method then expressing large complexities (more than doubly exponential, function like Ackermann) will be cumbersome. I do not understand your goal. — Evil, Dec 07 '16 at 23:21
A function that can compare any two algorithms whether they have the same time complexity or not, and produce a 'score' describing the effectiveness of one algorithm over the other. Like what I did with my polynomial method. Only applicable to all time complexities. A yardstick. — Tobi Alafin, Dec 07 '16 at 23:26
@D.W. $o$ notation may tell me $f(x)$ is more efficient than $g(x)$, but how much MORE efficient? I don't think I can get that from it. I'm ignoring constant factors. Asymptotic running time. When I mean efficiency, I'm talking purely about asymptotic efficiency. Worst case performance using RAM model of computation and the $O$ notation. Implementation isn't considered. I want a method to assign scalable consistent scores to an algorithm that operates in inverse ackermann time, and one that operates in ackermann time. — Tobi Alafin, Dec 07 '16 at 23:34
@Evil above, basically a method like mine, but one that applies to all time complexities. Basically a scorimg system for algorithms. — Tobi Alafin, Dec 07 '16 at 23:35

score 5 · Accepted Answer · edited Apr 13 '17 at 12:48

The concepts of asymptotic running time analysis and Landau notation already provide a way to compare the asymptotic running time of two algorithms, as long as (a) you are happy to ignore constant factors, (b) only care about asymptotics (limiting behavior as problem size gets large), and (c) you want to evaluate them according to their worst-case running time. It sounds like you are happy with (a), (b), and (c), so you should be happy with existing tools. They do tell you how much more efficient one algorithm is than the other. You just might not have appreciated yet how to fully use these mathematical tools and concepts, which is understandable.

In particular, if we can express the running time of algorithm A as $\Theta(f(n))$ and the running time of algorithm B as $\Theta(g(n))$, we already know how to compare them. We know that the ratio of their running time is $\Theta(f(n)/g(n))$. In particular:

If $f(n)/g(n) = \Theta(1)$, we know that their asymptotic worst-case running times are equivalent.
If $f(n)/g(n) = o(1)$ (i.e., $f(n) = o(g(n))$), we know that A is asymptotically faster, and we can measure asymptotically how much faster: it is faster by a factor of $\Theta(f(n)/g(n))$.
If $f(n)/g(n) = \Omega(1)$ (i.e., $f(n) = \Omega(g(n))$, or equivalently, $g(n) = o(f(n))$), we know that B is asymptotically faster, and we can measure asymptotically how much faster: it is faster by a factor of $\Theta(g(n)/f(n))$.

You might want to read How to come up with the runtime of algorithms?, How does one know which notation of time complexity analysis to use?, Sorting functions by asymptotic growth, How do O and Ω relate to worst and best case?, and Justification for neglecting constants in Big O. That will give you some essential background to help you understand how these concepts fit in.

(I'm simplifying a bit, and casting this at the level that it sounds like you're ready to absorb. Once you understand the basics, read Are the functions always asymptotically comparable? if you want to read some more precise statements -- but beware that right now this might just cause more confusion than it's worth.)

David Richerby · Answer 2 · 2016-12-08T08:44:12.507

2

All you're doing is comparing two functions by taking their ratio. The only reason you think this is restricted to polynomials is that you want to express the ratio as a polynomial. If you don't restrict the ratio to be a polynomial, you can relate arbitrary functions. (Actually, it's not restricted to polynomials since, e.g., if $f(n)=n^22^n$ and $g(n)=n2^n$, $f/g$ is still a polynomial.)

But, in general, the ratio of two polynomials isn't a polynomial; rather, it's a so-called rational function. For example, $(x^2+1)/x = x+\tfrac1x$, which isn't of the form $x^k$ for any real $k$, and which has no constant derivatives.

edited Dec 08 '16 at 08:44

answered Dec 08 '16 at 00:54

David Richerby

81,689
26
141
235

@TobiAlafin I ignored the part about the $m$th derivative being zero because, although it's something you state, it doesn't seem relevant to anything. It seems to be just a coincidence: sure, polynomials only have finitely many nonzero derivatives, but so what? In any case, the way you've written it, it's not even true. You assert that $r_i=n^k$ for some real $k$, and then claim that the $\lceil k\rceil$th derivative is zero. Well, take $k=\tfrac12$. The first derivative of $x^{1/2}$ is $\tfrac12x^{-1/2}\neq 0$. – David Richerby Dec 08 '16 at 08:41
1

It's as if you're saying "$x<y$ is an inadequate way of comparing real numbers. I'm going to use $x-y$ instead." But then restricting only to numbers where $x-y$ is an integer. Why? – David Richerby Dec 08 '16 at 08:45
Sorry -- replace zero with any other constant. No derivative of $x^{1/2}$ is a constant function. But why do you care about having a constant derivative? What "physical" meaning does that have? As you say, the constant is just a property of the leading coefficient and leading exponent of the polynomial. So why not just compare those directly instead of combining them into an arbitrary function? – David Richerby Dec 08 '16 at 10:49
2

Unless you can give an actual reason why looking at any derivative tells you anything relevant about the function, you shouldn't be considering derivatives at all. Anyway, please note that this isn't a discussion site; it's not a suitable place for fleshing out ideas. That sort of thing can be done in [chat] but it doesn't work in the Q&A part of the site. – David Richerby Dec 08 '16 at 11:17

Reasoning on Efficiency (2)

2 Answers2