Data Science Stack Exchange
Data Science Stack Exchange

Questions tagged [optimization]

In statistics this refers to selecting an estimator of a parameter by maximizing or minimizing some function of the data. One very common example is choosing an estimator which maximizes the joint density (or mass function) of the observed data referred to as Maximum Likelihood Estimation (MLE).

500 questions
9
votes
2 answers

Why does decreasing the SGD learning rate cause a massive increase in accuracy?

In papers such as this I often see training curves with this kind of shape: In this case SGD was used with a factor of 0.9 and learning rate decreasing by a factor of 10 every 30 epochs. Why is there such a large decrease in error when the…
geometrikal
  • 533
  • 1
  • 5
  • 14
6
votes
1 answer

Minimizing an upper bound of objective function

In many machine learning problems, you often have an objective function (e.g., cost, loss, error) that you want to minimize. Instead of directly minimizing this objective function, I sometimes see papers that derive the upper bound of the objective…
learning machine
  • 161
  • 6
5
votes
3 answers

Optimizing parameters for a closed (black-box) system

I am working on a problem that involves finding optimal parameter values for a black-box system. This system consists of 6 inputs and produces a single output. I can compare the returned values to observed values to determine whether the system is…
Ellis Valentiner
  • 205
  • 1
  • 8
5
votes
1 answer

Which Optimization method to use?

I have a non-function (not in closed form) that takes in a few parameters (about 20) and returns a real value. A few of these parameters are discrete while others are continuous. Some of these parameters can only be chosen from a finite space of…
Nitesh
  • 1,615
  • 1
  • 12
  • 22
4
votes
1 answer

Question on Scipy - Minimize. Adding additional constraints

I am trying to using scipy minimize function for the following optimization: V = np.matrix(pd.read_csv('V.csv'))` R = np.matrix(pd.read_csv('R.csv', index_col = 'Ticker'))` w0= list() for i in range(0, 84): w0.append(1/84) def…
JungleDiff
  • 217
  • 1
  • 5
4
votes
2 answers

What is the class of this optimization problem?

I have the following optimization problem: Find $\mathbf{w}$ such that the following error measure is minimised: $E_u = \dfrac{1}{N_u}\sum_{i=0}^{N_u-1}\lVert \mathbf{w}^Tx(t_{i+1})-\mathbf{F}(\{\mathbf{w}^Tx(t_j)_{j=0,i}\})\rVert$, $t_i \text{…
marianstefi20
  • 51
  • 4
4
votes
1 answer

how to make decision based on users reports

I have users reports about an accident, i want to know how to make sure that the number of reports is enough to take that accident as a true accident not a spam. My idea is to consider a minimum number of reports in a specific time interval, for…
honeyyy
  • 41
  • 1
3
votes
1 answer

How to optimize client's portafolio with analytical models?

I have a model in which we want to optimize the probability of an outcome depending on a election of some product (a personalized product for every client amongst three posibilities). The product is the second product elected after the same client…
Juan Esteban de la Calle
  • 2,292
  • 7
  • 28
3
votes
2 answers

Why RMSProp converges faster than Momentum?

Why is RMSProp in many cases converging faster than Momentum? Momentum: $$v_{dW} := \beta v_{dw} +(1-\beta)dW$$ $$W := W-\alpha v_{dw}$$ RMSProp: $$ S_{dw} := B \cdot S_{dw} + (1-B)\cdot (dW)^2$$ $$W := W- \alpha \frac{dW}{\sqrt{S_{dw}}}$$ Where…
Kari
  • 2,726
  • 2
  • 20
  • 49
3
votes
2 answers

Minimize absolute values of errors instead of squares

Calculating absolute values is much more efficient than calculating squares. Is there any advantage, then, to using the latter as a cost function over the former? Squares are easier to treat analytically, but in practice that doesn't matter.
Elliot Gorokhovsky
  • 133
  • 4
2
votes
1 answer

When is Non-Stochastic Global Optimization Preferable or Necessary?

Background I'm specifically referring to non-convex black-box optimization problems of the form: $ \text{min} f(\vec{x})$ $s.t. \ \ a_i\le x_i \le b_i \ \forall i\in \{1,2,...,n\} \ \ \ \text{and}\ \ \ \vec{a},\vec{b}\in \Bbb{R}^n $ Assume any…
Ben
  • 2,562
  • 3
  • 15
  • 29
2
votes
1 answer

Salesman problem with additional conditions and features

I'd like to specify the kind of a problem I encountered. I need to compose the best route for a car driver who goes to different cities. His aim to check in the most proper car park in a city taking into account arrival and departure dates and time,…
James Flash
  • 301
  • 1
  • 2
  • 8
2
votes
1 answer

Algorithm for campaign optimization (Digital Advertising)

Suppose i am running an Ad thru an Ad exchange A, and i have a set of campaigns running on it. I have The spend of the campaign. The budget allocated to it. The number of hours it took to exhaust it's budget. Total installs. Total Revenue. I want…
Benj Cabalona Jr.
  • 261
  • 1
  • 10
1
vote
1 answer

Optimal points of $f(x,y)=x^2 + y^2 + \beta xy + x + 2y$

I am self-learning basic optimization theory and algorithms from "An Introduction to Optimization" by Chong and Zak. I would like someone to verify my solution to this problem, on finding the minimizer/maximizer of a function of two variables, or…
Quasar
  • 113
  • 4
1
vote
1 answer

Find parameters to maximise output score

Not sure this is the right place to ask. Lets say there is a function f() where its implementation is unknown but it returns a score. I would like to get the highest possible score by modifying the input parameters. I also try to be better than…
Chris
  • 155
  • 4
1
2 3