@@ -115,7 +115,7 @@ A visual demostration of this pattern is given in \figurename{ \ref{fig:pixels}}
The goal is to encode the error between that value and the original value, save that, and use that to compress and decompress the image.
Even though a possibly larger integer may need to be stored, it is more likely that the guess will be correct or off by a small margin, making the distribution better for compression.
The approach of using the neighboring pixels for compression is not new, as evidenced by its use in ISO/IEC 14495-1:1999\cite{ISO/IEC14495-1} and ``CALIC-a context based adaptive lossless image codec''\cite{544819}, which were both written more than 20 years before the publication of this paper.
The approach of using the neighboring pixels for compression is not new, as evidenced by its use in ISO/IEC 14495-1:1999~\cite{ISO/IEC14495-1} and ``CALIC-a context based adaptive lossless image codec''~\cite{544819}, which were both written more than 20 years before the publication of this paper.
%This ``neighbor'' system is not as common as it should be, as it provides a base for simple implementation with high rates of compression.
Our final implementation differs from these methods, and others, in ways that we found beneficial for thermal images, and in ways others may find to be beneficial as well.
...
...
@@ -141,13 +141,13 @@ Normal compression systems work on thermal images, but since they are not optimi
\section{Related Work}
\subsection{PNG}
PNG is a lossless compression algorithm that also operates using a single pass system\cite{PNGoverview}.
The image is separated into several blocks of arbitrary size, which are then compressed using a combination of LZ77 and Huffman encoding\cite{PNGdetails}.
PNG is a lossless compression algorithm that also operates using a single pass system~\cite{PNGoverview}.
The image is separated into several blocks of arbitrary size, which are then compressed using a combination of LZ77 and Huffman encoding~\cite{PNGdetails}.
LZ77 operates by finding patterns in the data and creating pointers to the original instance of that pattern.
For example, if there are two identical blocks of just the color blue, the second one only has to make reference to the first.
Instead of saving two full blocks, the second one is saved as a pointer to the first, telling the decoder to use that block.
Huffman encoding is then used to save these numbers, optimizing how the location data is stored.
If one pattern is more frequent, the algorithm should optimize over this, producing an even smaller file\cite{PNGdetails}.
If one pattern is more frequent, the algorithm should optimize over this, producing an even smaller file~\cite{PNGdetails}.
The Huffman encoding in conjunction with LZ77 helps form ``deflate'', the algorithm summarized here, and the one used in PNG.
Our algorithm uses Huffman encoding similarly, but a completely different algorithm than LZ77.
...
...
@@ -155,11 +155,11 @@ LZ77 seeks patterns between blocks, while ours has no block structure and no exp
Ours uses the equivalent block size of 1, and instead of encoding the data, it encodes alternate data which is used to compress.
\subsection{LZW}
LZW operates differently by creating a separate code table that maps every sequence to a code.
Although this is used for an image, the original paper by Welch\cite{LZW} explains it through text examples, which will be done here as well.
Although this is used for an image, the original paper by Welch~\cite{LZW} explains it through text examples, which will be done here as well.
Instead of looking at each character individually, it looks at variable-length string chains and compresses those.
Passing through the items to be compressed, if a phrase has already been encountered, it saves the reference to the original phrase along with the next character in the sequence.
In this way, the longer repeated phrases are automatically found and can be compressed to be smaller.
This system also uses blocks like PNG in order to save patterns in the data, but instead by looking at the whole data set as it moves along, PNG only operates on a short portion of the text\cite{PNGdetails}.
This system also uses blocks like PNG in order to save patterns in the data, but instead by looking at the whole data set as it moves along, PNG only operates on a short portion of the text~\cite{PNGdetails}.
Ours, similarly to PNG, only looks at a short portion of the data, which may have an advantage over LZW for images.
Images generally do not have the same patterns that text does, so it may be advantageous not to use the entire corpus in compressing an image and instead only evaluate it based on nearby objects.
...
...
@@ -167,17 +167,17 @@ The blue parts of the sky will be next to other blue parts of the sky, and in th
\subsection{Similar Methods}
Our prior searches did not find any very similar approaches, especially with 16-bit thermal images.
There are many papers however that may have influenced ours indirectly or are similar to ours and need to be mentioned for both their similarities and differences.
One paper that is close is ``Encoding-interleaved hierarchical interpolation for lossless image compression''\cite{ABRARDO1997321}.
One paper that is close is ``Encoding-interleaved hierarchical interpolation for lossless image compression''~\cite{ABRARDO1997321}.
This method seems to operate with a similar end goal, to save the interpolation, but operates using a different system, including how it interpolates.
Instead of using neighboring pixels in a raster format, it uses vertical and horizontal ribbons, and a different way of interpolating.
The ribbons alternate, going between a row that is directly saved and one that is not saved but is later interpolated.
By doing this, it is filling in the gaps of an already robust image and saving the finer details.
This other method could possibly show an increase in speed but not likely in overall compression.
This will not have the same benefit as ours since ours uses interpolation on almost the entire image, instead of just parts, helping it optimize over a larger amount of data.
This paper is also similar to ``Iterative polynomial interpolation and data compression''\cite{Dahlen1993}, where the researchers did a similar approach but with different shapes.
This paper is also similar to ``Iterative polynomial interpolation and data compression''~\cite{Dahlen1993}, where the researchers did a similar approach but with different shapes.
The error numbers were still saved, but they specifically used polynomial interpretation which we did not see fit to use in ours.
The closest method is ``Near-lossless image compression by relaxation-labelled prediction''\cite{AIAZZI20021619}, which is similar in the general principles of the interpolation and encoding.
The closest method is ``Near-lossless image compression by relaxation-labelled prediction''~\cite{AIAZZI20021619}, which is similar in the general principles of the interpolation and encoding.
The algorithm detailed in the paper uses a clustering algorithm of the nearby points to create the interpolation, saving the errors to be used later in the reconstruction of the original image.
This method is much more complex, not using a direct interpolation method but instead using a clustering algorithm to find the next point.
...
...
@@ -192,8 +192,8 @@ To begin, the border values are encoded into the system, starting with the first
The values after that are just modifications from the first value.
There are not many values here and the algorithm needs a place to start.
Alternate things could have been done, but they would have raised temporal complexity with marginal gain.
Once the middle points are reached, the pixel to the left, top left, directly above, and top right have already been read into the system.
Each of these values is given a point in the x-y plane, with the top left at (-1,1), top pixel at (0,1), top right pixel at (1,1), and the middle left pixel at (-1,0), giving the target the coordinates (0,0).
%Once the middle points are reached, the pixel to the left, top left, directly above, and top right have already been read into the system.
%Each of these values is given a point in the x-y plane, with the top left at (-1,1), top pixel at (0,1), top right pixel at (1,1), and the middle left pixel at (-1,0), giving the target the coordinates (0,0).
Using the formula for a plane in 3D ($ax + by + c = z$) we have the system of equations
$$-a + b + c = z_0$$
$$b + c = z_1$$
...
...
@@ -246,10 +246,10 @@ z_0 + z_1 + z_2 \\
$$
.
The new matrix is full rank and can therefore be solved using \verb|numpy.linalg.solve|\cite{Numpy}.
The new matrix is full rank and can therefore be solved using \verb|numpy.linalg.solve|~\cite{Numpy}.
The x that results corresponds to two values followed by the original $c$ from the $ax+by+c=z$ form, which is the predicted pixel value.
Huffman encoding performs well on data with varying frequency\cite{Huffman}, making it a good candidate for saving the error numbers.
Huffman encoding performs well on data with varying frequency~\cite{Huffman}, making it a good candidate for saving the error numbers.
Figures \ref{fig:Uniform} and \ref{fig:Normal} give a representation of why saving the error numbers is better than saving the actual values.
Most pixels will be off the predicted values by low numbers since many objects have close to uniform surface temperature or have an almost uniform temperature gradient.
...
...
@@ -268,7 +268,7 @@ Most pixels will be off the predicted values by low numbers since many objects h
In order to adjust for objects in images that are known to have an unpredictable temperature (fail the cases before), a bin system is used.
The residuals from \verb|numpy.linalg.lstsq|\cite{Numpy} are used to determine the difference across the 4 known points, which the difference is then used to place it in a category.
The residuals from \verb|numpy.linalg.lstsq|~\cite{Numpy} are used to determine the difference across the 4 known points, which the difference is then used to place it in a category.
This number is the difference between trying to fit a plane between 4 different points.
If a plane is able to be drawn that contains all 4 points, it makes sense that the error will be much smaller than if the best-fitted plane was not very close to any of the points.
Something more certain is more likely to be correctly estimated.
...
...
@@ -312,10 +312,10 @@ Using a compiled language or integrated system instead of python will increase t
The issue with \verb|numpy.linalg.solve| was later addressed to fix the potential slowdown.
Calculating the inverse beforehand and using that in the system had marginal temporal benefit.
\verb|numpy.linalg.solve| runs in $O(N^3)$ for an $N\times N$ matrix, while the multiplication runs in a similar time.\cite{LAPACKAlgorithms}
\verb|numpy.linalg.solve| runs in $O(N^3)$ for an $N\times N$ matrix, while the multiplication runs in a similar time.~\cite{LAPACKAlgorithms}
The least squares method mentioned in this project also has a shortcoming, but this one cannot be solved as easily.
The pseudoinverse can be calculated beforehand, but the largest problem is that it is solving the system for every pixel individually and calculating the norm.
\verb|numpy.linalg.lstsq| itself runs in $O(N^3)$ for an $N\times N$ matrix\cite{LeastSquaredProblem}, while the pseudoinverse, when implemented, uses more python runtime, adding to temporal complexity.
\verb|numpy.linalg.lstsq| itself runs in $O(N^3)$ for an $N\times N$ matrix~\cite{LeastSquaredProblem}, while the pseudoinverse, when implemented, uses more python runtime, adding to temporal complexity.
This compression suffers when it is only used on individual images, which is not a problem for the use cases of this project.
The test images came from a camera that has 16 image sensors that work simultaneously.
