Gradient descent method to solve a system of equations

Question

I have the following system of equations and I need to solve this using gradient descent

$$\begin{cases} \cos(y-1)+x=0.5 \\ y-\cos(x)=3 \end{cases} $$

I understand more or less how to solve a singular equation but having troubles understanding how to solve systems.

These are the partial derivatives, but what is the next step?

$\cfrac{df_1}{dx} = 1$,
$\cfrac{df_1}{dy} = \sin(1-y)$,
$\cfrac{df_2}{dx} = \sin(x)$,
$\cfrac{df_2}{dy} = 1$

Gradient descent is a method for finding local minima of scalar-valued functions. It is not used for solving systems of equations, as far as I know. — , May 26 '19 at 14:07
You could formulate solving the system of equation as a nonlinear least squares problem, which is a nonlinear optimization problem, then apply gradient descent to that. — Mark L. Stone, May 27 '19 at 01:36
@Alexander: Try reviewing this: https://math.stackexchange.com/questions/1983713/gradient-descent-to-solve-nonlinear-systems — Moo, May 27 '19 at 02:20
The link by @Moo is the nonlinear least squares formulation I mentioned. — Mark L. Stone, May 27 '19 at 10:33

score 0 · Accepted Answer · answered May 27 '19 at 12:39

Here's my Swift code of solving this equation.

I know that this is not the best answer but that's all I have.

I found this code on C recently but I don't understand some of the things like what calculateM exactly returns and what algorithm it uses. So, if someone can explain this a little bit further that would be really great.

import Foundation

func f1(_ x: Double, _ y: Double) -> Double {
    return cos(y-1) + x - 0.5
}

func f2(_ x: Double, _ y: Double) -> Double {
    return y - cos(x) - 3
}

func f1dx(_ x: Double, _ y: Double) -> Double {
    return 1.0
}

func f1dy(_ x: Double, _ y: Double) -> Double {
    return sin(1-y)
}

func f2dx(_ x: Double, _ y: Double) -> Double {
    return sin(x)
}

func f2dy(_ x: Double, _ y: Double) -> Double {
    return 1.0
}

func calculateM(_ x: Double, _ y: Double) -> Double {
    let wf1 = (f1dx(x,y) * f1dx(x,y) + f1dy(x,y)*f1dy(x,y)) * f1(x,y)
        +
        (f1dx(x,y) * f2dx(x,y) + f1dy(x,y) * f2dy(x,y)) * f2(x,y)

    let wf2 = (f1dx(x,y) * f2dx(x,y) + f1dy(x,y) * f2dy(x,y)) * f1(x,y)
        +
        (f2dx(x,y) * f2dx(x,y)+f2dy(x,y) * f2dy(x,y)) * f2(x,y)

    return (f1(x,y) * wf1+f2(x,y) * wf2)
        /
        (wf1 * wf1 + wf2 * wf2)
}



var epsilon = 0.0
let maxEpsilon = 0.001
var iteration = 0
var w: [Double] = [10, 30]

repeat {
    let m = calculateM(w[0], w[1])
    var wNew: [Double] = [0, 0]

    wNew[0] = w[0] - m * (f1dx(w[0], w[1]) * f1(w[0],w[1]) + f2dx(w[0],w[1]) * f2(w[0], w[1]))
    wNew[1] = w[1] - m * (f1dy(w[0], w[1]) * f1(w[0],w[1]) + f2dy(w[0],w[1]) * f2(w[0], w[1]))


    if fabs(w[0] - wNew[0]) > fabs(w[1] - wNew[1]) {
        epsilon = fabs(w[0] - wNew[0])
    } else {
        epsilon = fabs(w[1] - wNew[1])
    }

    w[0] = wNew[0]
    w[1] = wNew[1]
    iteration += 1

    print("\nIteration №\(iteration)")
    print("x = \(w[0])   y = \(w[1])")
    print("Accuracy = \(epsilon)")
} while epsilon >= maxEpsilon

Gradient descent method to solve a system of equations

1 Answers1