Resources: overview, worksheets, linear least squares, linear interpolation.
When you have a few data points from a sensor, I haven’t found an easy way to calibrate. There are linear, quadratic, cubic, logarithmic curves.
It could be you enter data or draw how you want the curve to act, adding more control points if needed. It would output the equation if drawn, or different equations with the calculated error from the data given. Speed v accuracy being the trade off.
So much wasted time in Math. There should be a math test for people 5 years after they graduate high school. When the results are back you can pay me to consult how to improve it, part of the problem will be what you test on- that advice is free.
Starting point is linear with two points.
Equations: m = (y2- y1) / (x2 – x1) and with a point, y – y1 = m(x – x1)
In my case color sensor I wanted 1505 to output 0, and 361 to output 255.
m = (255-0) / (361-1505) = -0.223
coefficient = 255 / min-max
y – 0 = -0.223(x -1505) => y = -0.223+ 335
intercept = coefficient * -max
You can enter (1505,0)(361,255) into Wolfram and get an exact answer, Wolfram language may be a way to solve complex problems.