Colors in Forensics: The Analysis and Visualization of Forensic Data and Evidence (2024)

2.1.1 Properties of blood

Blood is often revered as the quintessential essence of life, symbolizing the enduring vitality inherent in the human body. It is readily obtainable, and variations in its constituent components serve as crucial indicators of an individual’s health status or, in forensic contexts, the age of blood trails [4].

Functionally, blood supports and sustains various bodily processes by facilitating the transport of essential substances. Mechanically propelled through the blood vessels by the rhythmic contractions of the heart, it traverses from the heart via arteries to organs and then returns via veins. On average, the human vascular system contains approximately 70–80ml of whole blood per kilogram of body weight. Comprising roughly 55% blood plasma and 45% blood cells, including platelets, red blood cells, and white blood cells, whole blood predominantly consists of red blood cells. Constituting around 96% of the blood’s mass, red blood cells (erythrocytes) serve as the primary carriers of oxygen absorbed from the lungs to organs and facilitate the transport of carbon dioxide from organs back to the lungs, thus facilitating cellular respiration [4].

The red color of blood stems predominantly from the protein hemoglobin, which constitutes the majority of red blood cell content. Hemoglobin comprises four subunits, each bound to an iron II complex known as heme. Central to the heme complex is an iron atom capable of reversible binding to oxygen molecules.

In the circulatory system, hemoglobin exists either in its oxygen-enriched form, oxyhemoglobin, or is deoxygenated during its journey back to the lungs as deoxyhemoglobin. Exposure to atmospheric oxygen outside the body gradually oxidizes oxyhemoglobin to methemoglobin, irreversibly converting the central iron atom to its trivalent state and binding water in place of oxygen. This oxidation process darkens the overall color of the blood. Over time, the concentration of methemoglobin increases, ultimately resulting in complete hemoglobin oxidation. As decay progresses, typically after 2–3 weeks, amino acids such as histidine tightly bind to the central iron atom, leading to the formation of hemichrome. This denaturation of blood pigment is accompanied by a color change from brown red to dark red or black.

Upon exiting the body, a drop of blood undergoes a gradual transformation in composition and subsequent color, transitioning from light red to deep black. The pace of this metamorphosis is contingent upon external factors, such as surface properties, temperature, or humidity [5].

2.1.2 Spectroscopic analysis of color change

Spectroscopy is a biophysical measurement method extensively employed across various forensic domains. It furnishes insights into the dimensions, configuration, architecture, charge, molecular mass, functionality, and kinetics of scrutinized macromolecules. This technique harnesses specific characteristics of light, enabling deductions regarding the condition of the analyzed sample.

More specifically, spectroscopy encompasses a collection of physical methods wherein the interactions between electromagnetic radiation and surrounding matter are observed and quantified. These interactions are recorded in the form of spectra using instruments called spectrometers. Spectra represent the intensity distributions of radiation as a function of wavelength, aiding in the identification of the substance under examination (Figure 1) [6, 7].

The aforementioned changes in blood, attributed to varying hemoglobin derivatives, can be monitored through blood spectra. These spectra exhibit evolving bands contingent upon the age and prevailing hemoglobin derivative. Notably, the Soret band, originating from the fundamental chemical structure of the heme group, exhibits a maximum intensity at approximately 425nm in young bloodstains. As stains age, this peak progressively shifts toward the ultraviolet range, with blood traces older than 3 weeks showcasing a Soret peak around 400nm. Additionally, young blood manifests peaks of oxyhemoglobin at 542 and 577nm. However, as blood ages, the proportion of methemoglobin increases, characterized by peaks at 510 and 631.8nm. Over time, oxyhemoglobin peaks diminish in intensity within the overall blood spectrum, giving way to methemoglobin peaks [8, 9].

In practice, forensic investigators collect blood samples from crime scenes and subject them to spectroscopic analysis. The spectral bands obtained are then mathematically compared with literature values to estimate the approximate age of the bloodstains. A detailed procedural elucidation of this methodology can be found in the chapter “Forensic Analysis of Bloodstain Color” from the book “Advances in Colorimetry” by IntechOpen (Figure 2).

Ultimately, it is imperative to underscore that spectroscopic recording of the age-related chemical alteration in the central iron atom of the blood pigment hemoglobin is conducted to estimate the forensically relevant timeframe during which the blood traces have been present at the crime scene.

2.2.1 Electrochemistry in dactyloscopy

The electrochemical analysis method facilitates the generation of high-resolution images of papillary ridges, thereby enabling the examination of sweat pores. This significantly enhances the recording quality of fingerprints and the underlying anatomical features. Furthermore, it opens up the possibility of translating sweat pore assessment from fundamental research into practical forensic applications. With this innovative technology alone, basic patterns, minutiae, and pores are displayed in high resolution. The varying compositions of biomolecules on the fingerprint allow for gender determination and differentiation between adults and children. Additionally, insights into contact with chemical compounds such as explosives, drugs, or medication can be gleaned. Importantly, this method preserves the structural integrity of the fingerprint.

The novel analysis method relies on an electrochemical reaction. Initially, the fingerprint is mixed with non-destructive marker substances and then stimulated (luminescence). The application of voltage induces the release of electrons from a chemical compound to a reactant. The electronically excited marker substance subsequently returns to its baseline state by emitting light (luminescence). However, if grease or sweat is present on the fingerprint carrier, the reaction is impeded. Consequently, only the spaces, that is, the finger grooves and pores, luminesce, create a highly detailed negative image of the fingerprint.

Alternatively, a positive image can be generated by treating the fingerprint with marker substances specific to the sought chemical compound (target substance). Subsequent application of the reaction partner results in luminescence only in areas where the desired compound is present. These may include narcotics, fire accelerants, or explosives. Relevant drugs within the scope of criminal law, such as cocaine, heroin (with the metabolite morphine or 6-monoacetylmorphine), codeine, methadone, and tetrahydrocannabinol (THC), can also be detected using this method.

For a comprehensive fingerprint analysis using this technique, the marker substance must target amino acids as the desired substance, causing the entire fingerprint to luminesce upon stimulation. Various substances can be employed for the basic electrochemical reaction. While luminol sometimes led to vigorous bubble formation on the trace, hindering analysis, ruthenium emerged as the ideal substance for the foundational electrochemical reaction.

These developed solutions culminated in the creation of an initial integrated solution comprising a modified camera and electrolytic cell, capable of implementing the desired functionalities. The high-contrast, high-resolution photos generated through this method could already be examined in the prototype using suitable image processing and evaluation programs (Figure 4) [14].

2.3.1 How a digital camera works/digital image collection

The design of a digital camera is modeled after that of the human eye, with various components of both systems carrying out similar functions.

Both the camera and the eye rely on external lenses to gather light, directing it onto the light-sensitive areas of their respective systems to achieve focus. Like the pupil and iris of the eye, the camera’s aperture adjusts to control the amount of incoming light based on the surrounding lighting conditions. Finally, the process of converting light into electrical signals by the light sensors in a digital camera mirrors the function of cone and rod cells in the retina of the eye. These cells transform incoming light into electrical impulses, which are then further processed [15, 16].

2.3.2 Image capture process

In the realm of photography, what the human brain intuitively controls in the eye must be replicated through intricate engineering processes. The journey of information from a subject in an image through the sensor system to a digital format involves several complex steps (see Figure 5).

Initially, the light reflected from the image subject is gathered by the camera’s outer lenses. These lenses are pivotal in concentrating incoming light and projecting a sharp image onto the image sensor.

To achieve clarity in the image, the camera adjusts the distance between these outer lenses to ensure the subject is in focus. Typically, this adjustment is managed by a small motor, guided by measurements from an integrated distance-measuring device. This computer-controlled autofocus mechanism significantly contributes to producing clear, detailed, and non-blurry images.

At this juncture, the paths of the camera and the eye diverge. While the eye can only focus on a small area of the image at a time, with the brain synthesizing the overall scene from this limited information, the camera must simultaneously focus on multiple pixels of the object [17].

The light collected by the outer lenses reaches the camera’s light sensor array, comprising millions of tiny photodiodes (pixels) that convert incoming light into electrical signals. To enhance image quality, each pixel is individually targeted by a corresponding microlens, with the number of microlenses aligning with the pixel count on the sensor. For instance, the iPhone 15 boasts a 48-megapixel camera, indicating a sensor array equipped with over 40 million lenses alone.

The functioning of a pixel resembles that of a photovoltaic cell, measuring the intensity of light it receives. This analog measurement (brightness) is then converted into a digital signal (brightness value). These sensors respond to light in the wavelength range of 190–1100 nanometers. This range encompasses ultraviolet (UV) and infrared (IR) light. Sensor sensitivity varies with material; while silicon sensors, common in most cameras, primarily respond to visible light, they can also react to other wavelengths, potentially causing visual noise if filtering is inadequate [18].

Since the light sensor detects the entire visible spectrum, filtering is necessary to capture not just brightness but also specific color values. This is achieved through millions of color filters distributed across the pixels, commonly organized according to the Bayer filter pattern [19]. The Bayer filter allows each pixel to capture only one primary color (red, green, or blue), enabling recording of both color and grayscale information. The Bayer filter arrangement is depicted in Figure 5.

Interpolation, also known as demosaicing or debayering, determines the color values of individual pixels on the image sensor. This process fills in missing color values through calculations involving neighboring pixels, resulting in each pixel having its own red, green, and blue values regardless of its original detection color.

However, interpolation may introduce artifacts such as blurred edges or color mixing, particularly in patterned subjects like a picket fence [20].

The outcome of this process is a RAW image, highly inefficient in terms of storage. Compression is often employed to mitigate storage requirements, albeit at the risk of data loss and image quality degradation.

2.3.3 Compression

To address the inefficiency of storing RAW images, compression algorithms like JPEG are commonly employed. The abbreviation “JPEG” stands for “Joint Photographic Experts Group,” the organization behind this standard. The JPEG compression algorithm reduces image file sizes by eliminating redundant information and compressing image data without perceptible loss in quality to the human eye. This process involves techniques such as discrete cosine transformation (DCT) and quantization [21].

However, compression involves loss of information (“lossy compression”), particularly in areas with intricate detail or low contrast. This can lead to visible artifacts like blocking or blurring, especially in heavily or multicompressed images. Such artifacts are scrutinized in digital forensic analyses, particularly in detecting image forgery [22].

AI-assisted methods, especially in restoring images compressed multiple times with JPEG, add complexity to digital forensic analysis, a practice known as antiforensics [23].

2.3.4 Color representation in digital space

A digital image consists of a finite number of discrete elements called pixels. Each pixel is identified by a specific, predefined position within the image and holds one or more finite, discrete values [24].

Mathematically, a digital image (S) is commonly represented by a matrix S=(s(x, y)). The rows (L) of this matrix are known as image rows, and the columns (R) are referred to as image columns. The individual elements within this matrix correspond to pixels. The variable x ∈ {0, 1, 2, …, L−1} denotes the index of the respective row, while the variable y ∈ {0, 1, 2, …, R−1} denotes the index of the corresponding column indicates. Thus, s(x,y) represents the value of a pixel at the associated location coordinates [25].

2.3.5 Digital color spaces

In the simplest color space, each pixel contains only one intensity value, resulting in grayscale images. These images are represented as a two-dimensional matrix S=(s(x, y)), where each matrix element is assigned a value from a set of gray values G, typically G={0, 1, 2, …, 255}. Alternatively, values can be assigned to an interval from 0 to 1, where 0 represents black and 1 represents white by default [25].

In the context of color images with multiple channels, each pixel is represented not by a single value but by several color channels (N). Thus, s(x, y) becomes an N-dimensional vector. Multichannel images are represented by a three-dimensional image matrix S(s(x, y, n)), where the new variable specifies the index of the respective image channel [25].

The RGB color space (see Figure 6, left), one of the most well-known, is depicted as a unit cube in a three-dimensional Cartesian coordinate system. It is based on the primary colors red, green, and blue. The grayscale values range along the main diagonal from (0, 0, 0) for black to (255, 255, 255) for white [24].

This representation maintains the additive color mixing property, where white can be achieved through the maximum intensity of mixing red, green, and blue. Any other color can be represented by a combination of these primary colors. The mathematical representation, assuming each primary color represents a channel (n), would be s(s(x, y, 3)) with s(x, y)=(gRed, gGreen, gBlue) (modified from [25]).

Another prominent color model is the HSV color model (see Figure 6, right), where H represents hue, S represents saturation, and V represents intensity, each ranging from 0 to 1. This model is often visualized as an inverted hexagonal cone, where (gH, 0, 0) represents black and (gH, 255, 0) represents white [24].

An advantage of the HSV color space is the ability to analyze and compare hue independently of brightness and saturation, which is beneficial for forensic analysis.

2.3.6 White balance

Color shifts within the pixel color values of an image can result in the image appearing tinted, often dominated by a particular color. This effect is particularly noticeable in night shots, where the orange light from street lamps influences the captured image, causing a shift toward an orange color range. To address this issue, the white balance method is employed. While the human eye can adapt to changes in lighting and resulting color shifts automatically, cameras often struggle with this adjustment. Professional photographers often use color charts with known color values to precisely correct color shifts. Without such reference charts, achieving accurate white balance can be challenging and may only be approximated.

One method used for white balance adjustment is the “Gray-world algorithm.” This method assumes that the average pixel value in an image is neutral gray (with a pixel color value of 128 in the RGB color space, represented as (128, 128, 128)) due to the normal distribution of colors. By analyzing the average color of the image, this algorithm estimates and corrects the color shift in the appropriate direction [26].

2.3.7 From the color to the case

Digital colors indeed hold significant importance in our daily lives, extending beyond the realm of multimedia. They also play a crucial role in casework, particularly in the development of specific evidence during criminal proceedings. For instance, colors can be utilized forensically to support various facts by comparing different hues to a desired color. Moreover, the ability to “make colors visible” is not merely fiction in certain cases.

2.3.8 Video analysis from rival motorcycle clans

The video material from Eisenbahnstrasse in Leipzig in 2017 depicts a volatile situation where two groups, identified by their distinctive black jackets, converge. The groups move toward each other, leading to a confrontation involving physical contact and gunfire. One individual is seen falling to the ground and remaining motionless. These groups are identified as the United Tribunes and the Hells Angels.

The footage, captured by a passer-by using a smartphone camera, suffers from poor quality and unstable camera work.

The initial question raised by the video material is: Which group entered the frame from the right, and which from the left?

The various groups are depicted in Figure 7, where a section of the image displays the logos on their jackets, albeit unclearly. Upon examining the original logos, it is difficult to definitively answer the question. However, in Figure 8, both logos are displayed clearly and scaled down to match the resolution of the image.

Simply comparing the shapes of the pixelated logos with the image sections taken may not provide sufficient information, but analyzing the colors can offer more insights. The standard Hells Angels logo incorporates the colors yellow and red. However, in the cellphone camera footage of the incident, various shades of gray are predominantly visible. Due to the low quality and resolution, the colors of individual areas are highly correlated, resulting in many pixels appearing gray. Nevertheless, the color spectra can illustrate the distribution and appearance of colors.

In Figure 9, the color spectra of both group logos are displayed. To achieve this, a section of the respective image was selected, focusing solely on the logo without including the surroundings or other objects in the analysis.

The color spectrum of the logo belonging to the group on the left exhibits a broad distribution across the colors red, green, and blue, ranging from less intense colors (appearing as black) to highly saturated areas. In contrast, the spectrum of individual colors in the logo of the group on the right extends less strongly into the intense color range. The differing characteristics of these spectra indicate that the logo on the left contains more color variation than the one on the right. Consequently, the spectrum on the left could be attributed to the Hells Angels group, as their logo typically incorporates colors as part of its standard design. Conversely, the logo of the group on the right displays much less intense colors, with the distribution mainly concentrated in the darker, black-appearing areas. Therefore, it could be associated with the United Tribuns logo, which typically appears in black and white without additional color.

Despite the limited information provided by the poor-quality video material regarding the colors of the logos, insights into color distribution can be gained through a color spectrum analysis. This method facilitated the assignment of the groups to their respective clans with a high degree of probability. It underscores the notion that just because colors may not be perceptible to the human eye in an image does not mean they are absent.

2.3.9 Identification of cars by color comparison from video material

In a scenario where a car drives on a road at night under low street lighting, surveillance cameras positioned across the street record the scene. Can digital forensics ascertain the color of the car and identify the make and model based on its paint job, especially when multiple models are available?

Specialized methods are employed to address this challenge, beginning with a color analysis of the car in the surveillance video. To accomplish this, one of the camera’s images is loaded into an image processing program to extract color values. The analysis step is depicted in Figure 10.

The figure illustrates that the red component of the image surpasses the green and blue components. Color determination is not solely reliant on visual inspection but is also documented using analytical methods. However, there are various shades of red within the area of car painting. The subsequent step involves comparing the target vehicle with other cars of similar colors to ascertain its specific shade. Based on previous process steps, the focus has been on the Fabia vehicle type from the Skoda brand. Figure 11 displays comparison models in various paint finishes within the red spectrum.

To generate meaningful comparison material, vehicles in the potential paint colors were driven past the original camera at night. The resulting video footage provides a more realistic basis for comparison. Upon comparing the underlying image or video material from the night of the crime with the comparison material, several differences emerge, including in the white balance. White values are closely linked to color temperature and brightness, factors that influence how objects are perceived by the human eye. For instance, cars may appear differently colored at various times of the day due to factors such as the position of the sun and atmospheric conditions like the refraction of sunlight by water molecules. The human brain naturally adjusts to interpret objects, such as a white sheet of paper or the pages of a book, as white under different lighting conditions.

However, a digital camera requires explicit instructions regarding the color temperature it should use, a process known as white balance. This adjustment was applied to the comparison video material.

Following adjustments to the video materials, the datasets of all comparison vehicles were prepared for comparison. Color matching of all vehicles with the original video material was conducted using a specialized image processing algorithm and quantitative assessment of color assignment. The results of these analyses, including correlations and differences, were summarized using an algorithm, with Figure 12 presenting the results graphically.

The diagram illustrates a notably closer match between the Corrida Red paint color and the vehicle involved in the crime compared to the other types. Through the utilization of specialized color comparison algorithms and the subsequent analysis connecting them, it can be inferred with a high probability that among all the compared paint colors, Corrida Red is the one most likely present on the crime vehicle.

2.3.10 The gold coin and the shoe

On March 27, 2017, the infamous Remmo clan stole the 100kg gold coin “Big Maple Leaf” from the Bode Museum in Berlin. As part of the planning of this audacious theft, various individuals were captured on surveillance footage at Hackescher Markt a few days before the crime. Among them was a person wearing highly distinctive shoes, which bore a striking resemblance to a pair of shoes identified by the police during their investigation (see Figure 13).

The pattern of the comparison shoe (Figure 13, middle) precisely matches the pattern on the suspect’s shoes (Figure 13, bottom left). However, a red stripe is clearly visible on the comparison shoe, which is not apparent in the video material. To investigate whether they are the same pair of shoes despite this discrepancy, comparative photos of the shoes worn by an LKA officer were captured using the same surveillance camera at the same location. This examination revealed that the surveillance camera loses some color information during recording and that, under these conditions, the comparison shoes at least appear similar to the suspect’s shoes.

To quantify the color changes caused by the surveillance camera, two umbrellas were set up at Hackescher Markt and filmed: one multi-colored and one white. The same umbrellas were also photographed under studio conditions in a photo laboratory. These images were then compared for color and brightness values (see Figure 14).

This investigation unveiled that the surveillance camera inadequately captures red tones in recordings (see Figure 14, segments 2 and 4).

In this instance, it becomes evident how different camera models perceive colors differently. Accuracy in coloring is often not the primary concern with surveillance cameras, as it is often compromised in favor of considerations such as storage space, frame rates, or resolution. This fact frequently comes into play, particularly in correspondence analyses.

2.4.1 Creation of artificial light

Light comprises electric and magnetic fields that propagate in waves, constituting electromagnetic radiation or electromagnetic waves. Typically represented as a sine wave, light is characterized by wavelength, frequency, amplitude, and phase. The human eye perceives colors within the wavelength range from approximately 380nm (violet) to 780nm (red) [27].

Artificially generated light, termed lighting, is produced through energy conversion. Specifications of a lamp include the color temperature, measured in Kelvin (K), and the luminous efficacy, expressed in lumens per watt (lm/W) [28].

Fluorescent lamps operate on the principle of gas discharge and fluorescence technology. The energy supplied ionizes mercury atoms within the lamp, causing collisions with electrons, which emit ultraviolet radiation. This radiation is absorbed by a fluorescent coating inside the lamp, converting it into visible fluorescent light [29].

Due to phase changes in the AC network, the light intensity of artificial lighting fluctuates between maximum intensity and an off state. This fluctuation manifests as flickering at twice the main frequency, typically around 100Hz in the European energy network [30].

2.4.2 European interconnected network

The public supply of electrical energy in Europe is facilitated by an interconnected system, comprising spatially neighboring and electrically connected transmission networks. Coordination and monitoring of this interconnected system are overseen by the association ENTSO-E (European Network of Transmission System Operators for Electricity) and its member transmission system operators [31]. As of March 2022, 36 European countries were integrated into this network system.

Since electrical energy cannot be stored sufficiently, energy consumption must align with production. Any imbalance between energy production and consumption leads to frequency deviations from the nominal frequency (fNenn) of 50.0Hz in the power grid. The primary cause of these deviations is block-by-block electricity trading, although demand fluctuations also influence grid frequency. These deviations are constrained within a range of ±200 mHz (millihertz), corresponding to a normal operating range of 49.80–50.20Hz. The frequency and amplitude of these deviations vary depending on the time of day or region. For instance, strong positive frequency deviations are typically observed before 5a.m., while significant drops in network frequency often occur in the evening.

Moreover, due to the 15 or 30-minute trading intervals in the European electricity grid, particularly pronounced fluctuations are expected between these intervals. There is a recurring pattern in the network frequency, characterized by a sharp drop at the start of a 15 or 30-minute interval, followed by an increase until the interval’s conclusion [32].

2.4.3 Network frequency analysis

Network fluctuations in the nominal frequency can be assessed using Electric Network Frequency Analysis (ENF). One of the pioneering works on ENF, dating back to 2005, is by Grigoras [33], which addresses network frequency artifacts in audio files recognized as forensically significant. The ENF criterion primarily involves measuring the mains frequency at a power source to establish a reference data set, extracting the ENF from audio material, and subsequently comparing both ENF tracks.

Initially, the research primarily focused on applications in the field of audio forensics. Grigoras’ approach outlines the extraction of an ENF track from audio recordings for timestamp verification. This involves downsampling to 120Hz and band-limiting the audio signal to a frequency range of 49–51Hz. The resulting ENF track is then visually compared with a reference database using Fast Fourier Transform (FFT) techniques [33].

Establishing a reference database for future analysis is crucial for classifying recorded ENF tracks. This entails selecting diverse measurement locations to capture frequency curves and reference data from various network areas.

2.4.4 Artificial light sources in videos

Video recordings from surveillance cameras, smartphones, digital cameras, and TV recordings are predominantly audiovisual, capturing both video images and accompanying audio tracks simultaneously. Additionally, digital video recordings are characterized by several parameters. These include:

1. Frame rate: the frame rate, measured in Hertz (Hz) or frames per second (fps), indicates the number of video frames recorded per unit of time.

2. Image size: this refers to the dimensions of the video frame, typically specified in terms of vertical pixels (columns) and horizontal pixels (rows), which determine the resolution of the video.

3. Color depth: the color depth specifies the number of color channels in an image, indicating the maximum number of possible color and brightness values per pixel.

The combination of these characteristics defines the video format of a recording. It is important to distinguish the video format from the data format used, which determines the technical structure and interpretability of the digital video data during processing. Typically, the data structure is organized as a container format containing separate audio and video content. This structure describes the compression methods or algorithms, known as codecs, used for encoding and decoding the audio and video content (Figure 15).

The global electronic shutter mechanism used here is called a global shutter. Compared to complementary metal-oxide-semiconductor (CMOS) sensors, charge-coupled device (CCD) sensors are more sensitive to light, have lower noise, and have a lower image readout rate. With CMOS sensors, pixels are exposed line by line from top to bottom and converted directly into a voltage signal and read out by an electronic circuit for each pixel element [31].

The “rolling” shutter mechanism used, known as a rolling shutter, increases the image readout rate, but the line-by-line scanning also represents a source of error, for example when recording fast-moving objects. If the entire pixel area is scanned line or column by line slower than the movement of the recorded object, a distorted image is created due to the time delay in scanning from the first to the last line. This error is called the rolling shutter effect [35].

2.4.5 ENF in imagery and “white wall” method

Through the analysis of audio components in video recordings, an Electric Network Frequency (ENF) analysis can also be conducted for video files. In cases where the audio track is absent or unsuitable for ENF analysis, the video content captured under artificial lighting can serve as an alternative. Garg et al. presented a method in [29] 2011 to detect ENF using optical sensors and video cameras. This approach broadened the scope of ENF forensic investigations to encompass the examination of image material within video files. Studies by Garg et al. in [29] revealed that fluctuations in light intensity, caused by emissions from fluorescent lamps commonly used in indoor lighting, are directly correlated with ENF.

Garg et al. introduced the “white wall” method, wherein a white wall serves as the constant backdrop of a video and is illuminated by a fluorescent lamp. The average intensity of video frames over a 10-minute period was calculated, and a bandpass filter with a frequency band of ±0.5Hz around the calculated “aliased” frequencies fa was applied. Subsequent bandpass filtering revealed the ENF signal in spectrogram displays. Spectrogram analysis was conducted using the short-time Fourier transform, with parameters set to a window size of 480 video frames (approximately 16seconds) and an overlap factor of 50%, facilitating the detection of ENF signals [29].

The results reported by Garg et al. underscore the necessity of calculating aliased frequencies f_a due to discrepancies between the fluorescent lamp’s light flicker frequency (f_l ≈ 100Hz) and the camera’s sampling rate (e.g., f_s = 30fs=30Hz). This mismatch violates the sampling theorem, resulting in aliasing effects. To translate recorded video intensities from the time domain to the frequency domain, Fourier transformation is employed. However, since Fourier transformation can only detect frequencies between 0 and the Nyquist rate (f_Nq), aliased frequencies f_a must be determined for the light flicker frequency of 100Hz [36].

The formula below, using the nominal light flicker frequency of 100Hz and the camera’s sampling rate is employed to calculate aliased frequencies [37]:

fa=∣fl−k⋅fs∣≤fs2,∃k∈NE1

The aliased frequencies indicate the frequency ranges within which the ENF track can be localized [38].

The occurrence of the rolling shutter effect, associated with CMOS camera sensors, further affects the accuracy of ENF analysis detection. A problem arises due to time delays affecting the sequential exposure and reading of image material [39]. Experiments with various cameras revealed that an idle period follows the sequential line-by-line reading of video images before the next video image is read out [40]. Consequently, the ENF signal is not continuously captured. It was also observed that sequential image capture can contribute to an increase in the effective sampling rate, as multiple points in time are captured within a single video image [39]. This directly impacts the camera sampling rate of CMOS-based devices.

Adjustments to the sampling frequency are necessary to account for this phenomenon [41]:

fscmos=F1×h⋅fsE2

Examining the formula, it becomes evident that the product of the height F_1×h (in pixels) of a video image and the video frame rate f_s (in fps) yields an increased sampling rate fscmos for videos recorded with CMOS sensors [41].

2.4.6 Comparison of network frequencies with databases

A comparison of the data between the recorded reference data and the data determined from the audio or video content can be performed by calculating the correlation coefficients. The correlation coefficient is a measure used in signal processing to quantify the similarity between two spatial or temporal functions, X and Y. Depending on whether the correlation coefficient is closer to +1 or−1, it indicates a positive or negative correlation between the data series. A positive correlation suggests that the measured values in the comparison file and the original file increase or decrease simultaneously. Conversely, if the correlation coefficient is closer to 0, it indicates little or no correlation.

To compute the correlation, the two signals under consideration, the one from the reference database and the ENF signal recorded from the video, are compared. To determine the time of recording in the reference data set, the shorter signal, the network frequency in the video recording, is systematically shifted relative to the longer signal, the network frequency of the reference database. The peak of the correlation function then indicates the maximum similarity between both signals.

Establishing the temporal reference of the network frequency in the reference recording enables the classification and determination of the recording time of the video, akin to a forensic timestamp of a video file.

The forensic utility of ENF analysis can be categorized into the following areas of application:

1. Determination of recording times through temporal analysis of reference data.

2. Identification of the recording locations by correlating different network connection areas.

3. Assessment of continuity/authenticity by identifying the presence of ENF signals.

2.4.7 Conclusion

With the increasing prevalence of video recordings generated using artificial intelligence or manipulated multimedia data, such as deepfakes, ENF analysis emerges as a crucial tool for verifying their authenticity [42]. Detecting deepfakes and other manipulations of audiovisual media files is essential in the fight against terrorism [43]. Additionally, these techniques are invaluable for solving various criminal acts related to video material. Examples include crimes involving ransom demands, kidnapping scenarios, attacks on surveillance systems, and the widespread distribution of child p*rnography video files [43].