Abstract
In the fast-moving consumer goods (FMCG) industry, current labelling solutions have challenges to meet the track & trace requirements. Currently, FMCG items use mainly paper-based self-adhesive labels with traditional barcodes. These labels are low priced and technically easy to produce and deploy. The shift towards advanced solutions, like radio frequency identification (RFID) or near field communication (NFC) tags, still does not offer a good enough cost/benefit ratio. These advanced solutions have a high unit price or require costly changes in production lines. Still, the industry recognizes the possibilities of smart tags. Recent research has shown that functional inks can operate as cheap sensors. However, more research is needed to take functional inks into the operational FMCG environment. This paper presents one technical solution for an FMCG smart tag. The proposed smart tag builds on traditional QR-Code and Datamatrix markers, printed with standard inks. However, it also has functional ink embedded inside the marker as a sensor. This research experiments how embedding impacts the overall performance of the smart tag decoding. And if the CIEDE2000 color difference algorithm can calculate the state of the sensor. Three different parameter combinations, CIEDE2000(1, 1, 1), CIEDE2000(2, 1, 1), CIEDE2000(2.76, 1.58, 1), and their accuracy are compared. Experiments show that the proposed approach does not negatively affect the decoding performance. And that a color comparison can detect sensor states, especially when the functional ink has high enough color intensity. Between different parameters, CIEDE2000(2.76, 1.58, 1) performed best, especially in the low-intensity test. However, some future research needs to address absolute color value detection and the accuracy of color recognition; especially when if the color has low intensity.
Similar content being viewed by others
Avoid common mistakes on your manuscript.
Introduction
From the day mobile devices have had cameras, they have been used to decode markers. The most popular markers today are QR-Code and Datamatrix. These markers can be found in many items starting from fast-moving consumer goods (FMCG) and going all the way to industrial and medical usage. According to Tiwari (2016), the first one-dimensional (1D) marker was invented in the 1960s. Since markers have evolved from 1D barcodes to multi-color two-dimensional (2D) markers (Fig. 1) [56]. Marker development has followed other technologies that relate, like cameras and image vision. Currently, there are multiple marker types, some more general, some more use-case specific.
This research extends standard QR-Code and Datamatrix markers to smart tags. Smart tags primary function is to provide information to its user regarding the status of item [38]. Printed non-electronic smart tags function by communicating through the physical senses (human vision) or machine vision.
Functional inks are used to extend standard QR-Code and Datamatrix markers. These inks form the part of the marker which acts as an active sensor by reacting to environmental variables like humidity, temperature, or light. Functional inks are suitable to the normal label print process of markers. The chemical structure of the functional ink determines how and on which environmental variables ink reacts.
As summary, this research looks into solving following research questions:
-
How to embed functional ink inside the QR-Code or Datamatrix?
-
How does the embedding impact the decoding performance of QR-Code or Datamatrix?
-
Can CIEDE2000 color difference algorithms accurately recognize state of the functional ink?
The rest of the paper is organized as follows. Section “Related Work” goes through related work. Section “Materials and Methods defines what smart tags are, how label printing works, introduces use-cases that relate to smart tags, and looks into the proposed technical approach. Section “Experiments” discusses details of experiments, and in Section “Results and Discussion”, results of the experiments are discussed. Finally, Section “Conclusions” concludes the work, followed by future research topics in Section Future Work.
Related Work
Relevant past research work has focused on exploring possibilities of intelligent packaging in the FMCG industry. As part of intelligent packaging smart tags have been seen as a possibility to track items and their status. For example, this research has been done in the context of food items by Yam [60], Mohebi and Marquez [34], Realini and Marcos. [43], Kalpana et al. [21], and Fuertes et al. [8].
Related research has developed new advanced markers. These markers have one difference from traditional markers. Where traditional markers use black and white values to present binary values, advanced markers use a limited spectrum of colors. Parikh and Jancke proposed an approach to recognize multiple colors in 2D color barcodes [36]. Bulan and Sharma proposed using dot orientation, with colors, to encode data into high-capacity barcode [4]. Grillo et al. proposed a new barcode that uses colors to include more data [13]. Bulan and Sharma proposed a barcode that uses the cyan, magenta, and yellow colorant separations to enable high-capacity barcode [5]. Subpratatsavee and Kuacharoen proposed a new barcode that reduces the physical space needed for the barcode. In their research, Subpratatsavee and Kuacharoen present the design and the implementation of a high-capacity two-dimensional barcode [50]. Taveerad and Vongpradhip proposed color QR-Code to hold more data through a novel encoding concept [54]. Common to all these approaches is that in theory, black & white marker data can only be translated into 1s and 0s when decoded. However, colors, or shades, can be translated to as many values as possible to identify [41]. Most of these past studies relate to using colors to encode more data into barcodes. Or how to change data that barcode contains through colors. John and Raahemifar provided an overview of color barcodes. They also proposed a binarization and grou** algorithm to encode data to form a color barcode [20]. Wasule and Metkar took into account the intensity variation which occurs while decoding colored barcodes and proposed an approach that will increase the capacity of barcodes beyond threefold; in their work, they used quantization of grey levels [59]. Ramalho et al. used the concept of super-modules to encode more data into QR-Code but also to make QR-Code more secure [42].
Functional inks and mainly their properties have been studied during recent years. Gao et al., in their research, focused on time-temperature indicators (TTIs) data modelling [9]. Li and Chen used two different functional inks in their proposal as a new Dynamic and Sensitive Barcode (D&S), D&S can react to environmental state change [27]. Chen et al. used a vice-versa approach and made the paper reactive to environmental changes, and they also used smart devices to decode color information [6]. Kulčar et al. focused their research on the properties of functional inks regarding state changes [24].
One of the most relevant research around functional inks and smart tags is a study by Gligoric et al. In this research, authors defined possible approaches for smart tags [12]. Quite similar work was done by Hakola and Vehmas [14]. However, these studies mainly focus on uses-cases and properties of functional inks, and only briefly define technical frameworks for ink-based smart tags
This research uses CIEDE2000 (KL, KC, KH) algorithm for color difference calculations. CIEDE2000 has been used in many studies where color difference is matched against human perception or in general color comparison. This work has been conducted for example by Taoa et al. [52], where authors used the algorithm to recognize leaf color differences. By Nguyen et al. [35] in the context of rice color recognition and by López et al. [30] for mortar color differences. Usage of KL = 1, KC = 1, and KH = 1 parametric values is the most common approach when using CIEDE2000 [11, 31, 37]. Past research has also achieved good results with KL = 2, KC = 1, and KH = 1 parametric values [10, 32]. The third set of parametric values used in this research is KL = 2.76, KC = 1.58, and KH = 1, which has been used in the past, especially in digital and printed images [29].
The past research approaches smart tags from different point-of-view; they rarely consider the actual FMCG label printing process. And previous color difference research has focused on color difference calculations; especially, in use-cases where color comes from a physical object, or colors have a different hue.
Contributions of this research work are:
-
(a)
Technical approach to extend standard QR-Code and Datamatrix markers to smart tags, without significant impact on marker decoding performance.
-
(b)
Verify the suitability of embedding functional inks into QR-Code and Datamatrix markers.
-
(c)
CIEDE2000 algorithms’ parameter comparison with printed colors.
Materials and Methods
Labels
Smart tags have been defined in the past research in various ways. They might refer to radio-enabled electronic devices, e.g., [62], radio frequency identification (RFID), or near field communication (NFC) tags, e.g., [1]. Smart tags have also been considered as printed electronics, e.g., [45].
In this research, smart tags are standard two-dimensional markers with added intelligence from functional inks. Smart tags are printed on self-adhesive labels, and finally glued to FMCG products. This paper does not take into account labelling/printing that is done directly on items, like laser marking. According to Kirwan (2012), the invention of self-adhesive labels was in the mid-1930s. Labels were back then used to apply price and decoration on store items. Currently, the FMCG industry uses labels to add value to a product item during its life-cycle. The latest development of labels has made them smart, smart-active, or smart-intelligent (Fig. 2). Smart labels have various usages like tracking products, monitoring their temperature, and indicating food freshness [23].
Smart tags have the functionality to react to environmental changes and identify individual items, rather than presenting only static content. This categorizes smart tags as customized labels [14]. Smart tags defined in such a way are part of intelligent packaging, an emerging technology in the FMCG. Intelligent packaging uses smart tags as the communication function between the package and the user. Smart tags and their information help in the decision-making related to the item, to achieve the added value, like enhanced food quality, user experience, and safety [60].
Label Printing Process
Labels with smart tags are either (a) pre-printed in print houses or (b) printing occurs just before they are applied (in-house) [49].
Depending on the chosen label printing facility, pre-print or in-house, multiple technologies can be used to print labels.
-
Rotary and semi-rotary letterpress.
-
Flexographic printing.
-
UV-flexographic printing.
-
Screen printing.
-
Offset printing.
-
Rotogravure printing.
-
Thermal printing.
-
Laser and inkjet printing [23].
All listed technologies are suitable for the pre-printing of labels. For labels printed in the production line, thermal, laser, and inkjet printing are plausible. These non-conventional technologies can also print customized information that varies per label. It is also possible to mix conventional and non-conventional ways of printing; this is called hybrid printing [23].
Electronic/non-electronic labels can occupy as much as 50% of the package costs. However, in most low-priced products, like food, the price of the package should not exceed 10% of the total costs of the product [58].
Functional Inks
Functional inks can report exposure to environmental influences by switching between two states of optical properties. The state of the ink and its absolute value depends on the current properties of the used ink. Functional inks can ultimately appear either in an active (1) or non-active (0) binary state. Switch between states is relative to physical influences. For example, water/humidity (hydrochromism), temperature (thermochromism), and the intensity of light (photochromism) can change the state of the functional ink [15].
Depending on the environmental influence, and properties of the ink, change between two binary states occurs in a specified time. During this period, functional ink can also have values between 0 and 1. Optically visible change between states can either be a change between colors or color-changing its intensity. Also, depending on the chemical compound of the ink, change can be reversible or irreversible [3].
According to Zabala et al., the change in functional ink can represent accumulated exposure (total), or if the ink has exceeded its activation point [61]. Therefore, functional inks can track continuous exposure during the whole life-cycle, or exposure to an environment lower/higher than the threshold set.
Functional inks in this research are compatible with the high-speed printing process (conventional, non-conventional, and hybrid). And to meet labelling cost requirements, the price of the functional ink is close to the price range of standard color inks. The advantage of the smart tag with functional ink is that there are no electronics in it. This makes the manufacturing technically less complex than RFID/NFC-based smart tags. The advantage is also in recycling, as some regulations define RFID/NFC-based tags as electronics. Using functional ink raises the possibility to design innovative products for markets not yet addressed by the electronic tags [12]. Functional inks are suitable for most printing methods presented in the previous chapter, including flexographic, gravure, screen, and inkjet [28].
When smart tags are developed for the needs of the food industry, they should not impose thread to items inside packages or when in direct contact with food items. There are three main ways of food safety can be compromised by the ink used in labels/packages [46]:
-
Migration, components of ink pass through the substrate
-
Invisible set-off—transfer of components of ink from the printed side to the food-facing side, for example when packages are stacked.
-
Gas-phase transfer of ink components via the air in the packaging to food.
Some functional inks are suitable for short- or long-term direct food contact (DFC). These inks meet regulations set by authorities like EuPIA (European Printing Ink Association) and U.S. Food and Drug Administration (FDA). However, some functional inks can only be used in non-food contact. Past research has also developed options for this like impermeable barriers/papers, which can be used when printing smart tags [61]. As this paper only focuses on proposing a general approach, research of these options is left for future research.
Use-Cases for Smart Tags in FMCG Industry
Fast-Moving-Consumer-Goods are perishable packaged products purchased and consumed by all members of society. FMCG cover items like food and beverages; often, these goods are also considered as Consumer Packaged Goods (CPG). One of the main purposes of the product packaging is to communicate important information about item(s) in the package to the user. Information user gets through labels, which might relate to the ingredients/material of the product. However, the label can also contain other information, like nutritional facts. Packages, and their labels, look to meet customer experience expectations. However, some of their functionality is also responsible for fostering a circular economy. Part of this is cradle-to-grave tracking and actions that relate. Typical life-cycle (cradle-to-grave) of an product is illustrated in Fig. 3.
As shown in Fig. 3, intelligent packaging links to all phases of the product life-cycle. In this context, smart tags should deliver various information to the user. This information contains either data about the current status of the product or information that relates to the product’s historical status. Depending on the life-cycle phase, user might be interested in a lifetime, temperature history, freshness status, package condition, authenticity, or something else [23].
Smart tag-related research has recognized suitable use-cases which directly or indirectly contribute to meeting the needs of consumers, producers, and manufacturers [7]. These uses-cases are classified as follows:
-
Manufacturing (Raw material, Product manufacturing, Packaging).
-
Distribution & Consume.
-
Recycle [38].
Listed use-cases are related to item identification and providing information about the item to the current stakeholder or user. When the packaged item has a smart tag on its label, the item has intelligent packaging. The intelligent package is “a packaging that contains an external or internal indicator to provide information about aspects of the history of the package and/or the quality” [44]. Some use-cases are more suitable for intelligent packaging than others.
As printed smart tags belong to the same category as RFID tags’ most use-cases can be derived from RFID/NFC use-cases [12]. Printed smart tags lack a radio interface, so the decoder must have a line of sight and relatively close distance. However, printed smart tags support a visual interface that low-cost RFID/NFC do not currently have. Generally, the costs of the smart tags are part of the packaging costs.
Technical Approach for the Smart Tag
Smart tags developed for the FMCG, must be able to: (a) avoid any false negatives (samples that seem safe but are dangerous) and (b) they should have as few false positives (samples that seem unsafe but are healthy) as possible.
If a functional marker detects false negatives, it provides information that the item is safe, although the reality is vice versa. False positives are not that crucial, but providing false positives weakens the trust towards the system. False positives provide information that the item is unsafe even if they are not. The price of the marker is also a meaningful factor and how the user can decode the marker [39].
Intelligent packaging use-cases are dependent on two features. First is a unique identifier, which provides a way to trace items’ history and link life-cycle events of the item into databases or other services [38]. The second feature is a use-case-specific sensor printed with functional ink. Therefore, markers must be printed with two different inks, one that is static (unique identifier) and one that can react to environmental changes (sensor) [51].
The following three methods can be when extending markers with functional ink [51]. These are:
-
(a) Functional ink placed outside of the marker, a solution used by many of the existing smart tags. This increases the required physical print area and might make decoding of the functional ink state complex.
-
(b) Functional ink changes markers’ data, and this solution is more advanced and requires specific and matching functional inks and compatible marker contents.
-
(c) Functional ink placed inside of the marker, a solution presented in this paper.
In the proposed approach, functional ink is placed inside the marker without disturbing the reserved cells, like a timing pattern. This does not exceed markers’ error correction capability. However, due to ink misplacement or dye growth, the approach uses functional ink intensity satisfying requirements for symbols set by GS1 standard (bar code symbol print quality test specification—two-dimensional symbols) [16]. In these requirements, the contrast between the symbols black and white cells should not be lower than 40%. Placing functional ink inside marker also affects modulation criteria, which compares local contrast to global contrast. As functional ink area disturbs the global threshold, it might affect the probability of incorrect cell color identification. In such a case, base markers’ capability to recover from error/destroyed data is used while decoding. When using QR-Code or Datamatrix as the base marker, around 30% of destroyed data can be recovered [17, 18].
The proposed approach has the following advantages: (a) it does not occupy more physical space than adding standard QR-Code, (b) QR-Code can work as call-to-action, and (c) it is technically suitable for the current label printing processes. Technically approach might be suitable for use-cases with more than one functional ink (sensor), but more research is needed.
Figure 4 shows proposed approach in QR-Code. In this approach, the sensor area would be defined by rectangle
where n is the count of cells marker has in vertical/horizontal direction and indexes of the cells start from top-left corner where index is (0,0) and bottom-right corner has index (\(n-1,n-1\)). When sensor data are occupying such a rectangle, it does not occupy more space than 24 cells (\(\tilde{1}3\%\)) of 189 data cells in the smallest possible Qr-Code (21\(\times\)21) [17]. Some research needs to be done in the future to explore if this amount of sensor cells is enough for accurate color recognition. Theoretically, the sensor could occupy more area within QR-Code. However, this might affect the decode performance of the QR-Code. Especially performance could be lower in challenging environments if more errors occur in other parts of the QR-Code [26]. Using only one row for the sensor might affect sensor data decoding, especially in small QR-Codes. This is because the QR-Code generator algorithm optimizes the ratio between white and black cells, but might sometimes generate rows where black/white cells have the majority [17].
Functional ink inside the marker is straightforward to locate. When decoding a standard QR-Code, an algorithm finds coordinates of each cell. As the algorithm knows the location of each cell, it can also define the coordinates of the sensor area. In the proposed approach, standard QR-Code scanner applications can decode contents of the QR-Code, but cannot decode sensor area information.
Decoding of the sensor value happens with a special algorithm. This algorithm calculates the difference between white and black cells’ color values within the sensor area. Then, the difference is compared to the maximum difference, calculated in the same way but from the reference area. The color difference is calculated by CIEDE2000(KL, KC, KH) formula [31]. The performance of different parameters is compared later in the experiments section. Figure 5 shows the process chart of the algorithm. The sensor is considered active if its value is over 10% of the reference maximum difference.
Figure 6 shows the reference area (REF) and sensor area (S1). These areas work as sources for the calculation of the white-black or sensor-black difference. The location of the reference area is two rows above the sensor area because of possible functional ink spreading or misplacement. Placing the reference area further from the sensor are might alter it to ambient lighting changes like shadows, different to ones that the sensor has.
Even though in Fig. 4, it shows sensor in the QR-Code marker, the approach should apply to other markers with the error correction capability, like Datamatrix. As such, the addition of the sensor affects the general performance of the marker, discussed in the next chapter. Depending on how much data are recoverable from damaged markers (Datamatrix or QR-Code), height of the sensor area can vary in different sized markers. In this research, sensor area is always two rows in height.
Experiments
This section focuses on the practical experiments with the proposed approach. The focus of the first experiment is to validate how sensor embedding impacts the decoding performance of the marker. The focus of the second experiment is to test the calculation of the sensor value in simulated use-cases. Finally, the third test experiments with three real-life markers using actual functional ink.
Past research has shown that the size of the marker should be larger than 10\(\times\)10mm and have at least 4 pixels (px) per cell [40, 53]. Based on this information, for the first two experiments, three different markers in size 20 mm \(\times\) 20 mm were printed with an office laser-jet (Canon ImageRunner C5535i). Printing was done in 300dpi to standard office A4-paper (Canon Black Label Plus 80 g/m\({^2}\)).
Markers had different data to represent different item-level tracing options:
-
(a)
EAN/UPC-13 data [47], 13 digits, illustrated as 25\(\times\)25 sized QR-Code
-
(b)
EPC data, 26 digits [55], illustrated as 29\(\times\)29 sized QR-Code
-
(c)
UUID data [57], 36 digits, illustrated as 37\(\times\)37 sized QR-Code.
Using different sizes is to experiment with how the proposed approach performs when there is a different amount of pixels per cell.
The third experiment uses real-life markers generated with the proposed approach. These markers were printed with the following functional inks:
-
(1)
LCR Hallcrest cold activated thermochromic ink, clear to green CMYK (22,7,17,0) at 7 \(^\circ\)C, 38 cm\({^3}\)/m\({^2}\), 1 printed layer [25]. Printed using flexographic printing.
-
2)
Datamatrix with SICPA heat activated thermochromic ink, clear to magenta CMYK(0,100,0,0) at 26 \(^\circ\)C, 38 cm\({^3}\)/m\({^2}\), 1 printed layer [48]. Printed using screen printing and flexographic printing.
-
3)
Datamatrix with Sunlase TDS sulfuric acid reactive ink, clear to black CMYK(0,0,0,100), 38 cm\({^3}\)/m\({^2}\), 1 printed layer Printed using screen printing.
The third experiment has also the difference that it uses Datamatrix codes where functional ink is embedded in the same way as in QR-Code used previously.
QR-Code Integrity Experiment
The first test experimented with two versions of each marker (a) version where sensor area was transparent (sensor off), in Fig. 7 top row. And (b) version where sensor was totally black (sensor on), in Fig. 7 bottom row.
Latter markers simulate a situation where the sensor destroys all white cells within the specified data region, and error correction recovers these data. This experiment looks to validate that adding a sensor does not prevent the decoding of the marker.
In this experiment, iPhone 7 and iPhone 11 Pro were used, with a standard camera application, to decode QR-Code markers from different distances. Controlled environment, 25 cm \(\times\) 20 cm \(\times\) 40 cm sized white box, was used for the test. The test environment had a moving sledge for distance control and adjustable LED lights for ambient lighting control. Decoding distance was measured by moving the sledge to the furthest distance where decoding of the marker happens continuously. In this experiment, decoding distance was tested in two ambient light levels, 400lx (office lighting) (Table 1) & 15lx (dark environment) (Table 2).
Sensor Value Calculation Experiment
The second experiment focuses on testing if the algorithm defined in the previous section can identify sensor states when different sensor colors are used. Second experiment uses same marker sizes and contents as the first experiment. However, this time with different sensor colors and their intensities. Four pure colors magenta (\(C = 0.0, M = 1.0, Y = 0.0, K = 0.0\)), cyan (\(C = 1.0, M = 0.0, Y = 0.0, K = 0.0\)), yellow (\(C = 0.0, M = 0.0, Y = 1.0, K = 0.0\)) and black (\(C = 0.0, M = 0.0, Y = 0.0, K = 1.0\)). And three mixed colors red (\(C = 1.0, M = 1.0, Y = 0.0, K = 0.0\)), green (\(C = 1.0, M = 0.0, Y = 1.0, K = 0.0\)) and purple (\(C = 1.0, M = 1.0, Y = 0.0, K = 0.0\)) were used.
For each color, three marker sizes were printed with different intensity (100%, 80%, 60%, 40%, and 20%) levels. Markers were captured, with iPhone 7, Nokia TA-1032, and iPhone 11 Pro to images from two distances 10 cm and 20 cm, in the same ambient light levels as in experiment one. Aside from capturing markers in a controlled environment, images were captured in random contexts, including daily environments like fridges, office tables, and store shelves. This totalled 1260 images for the second experiment, and examples of photos used in this experiment are shown in Fig. 8.
In the second experiment, QR-Codes were decoded with a custom Python application [19]. This application used algorithm shown in Fig. 5 to recognize sensor state. In the experiment, three CIEDE formulas with different parametric values for KL, KC, and KH were used to calculate color difference. These formulas were CIEDE2000(1,1,1), CIEDE2000(2.76,1.58,1), and CIEDE2000(2,1,1).
Tables 3, 4, and 5 show the results of the second experiment. Results are separated into three tables depending on parametric values. The first column shows sensor color, and the following columns show the lowest and highest value of the sensor. These results can be used to estimate the usability and accuracy of the algorithm and its parameters.
The third and final test experimented with smart tags printed with real functional inks. These smart tags were based on Datamatrix and photographed in a real-life environment with iPhone 7, Nokia TA-1032, and iPhone 11 Pro. First before exposing them to a reactive environment. After exposure, markers were photographed again. Fifty images were captured for each marker and sensor state, totalling 300 images. Examples of photos are seen in Fig. 9.
The same Python application as in experiment 2 was used to decode markers. Table 6 shows the results of the third experiment. Columns show different CIEDE200 parameters, and values range from decoding. On the rows are different marker states
Results and Discussion
Based on the results from the first experiment, adding a sensor inside the marker reduces the decoding distance only slightly. Distance is decreased more in a dark environment, but the absolute change in the decoding distance is small. In everyday use, this does not have a significant impact. Markers with more data are affected less as the sensor occupies a relatively smaller area from the data area.
In the second experiment, simulated sensors with different intensities [0%, 20%, 40%, 60%, 80%, and 100%] were decoded with the proposed approach. The second experiment included sensor value calculations with the CIEDE2000 algorithm. Three different parametric (KL, KC, and KH) values were tested, (1,1,1),(2.76,1.58,1) and (2,1,1). Results show that when sensor intensity is 40% or more, all parametric combinations can recognize the state of the sensor. When intensity is lower (20%), calculations are not working accurately, and incorrect states are sometimes recognized. Especially, algorithm has challenges with the yellow ink. With higher intensities, the algorithm gives values over 0.1. This can be considered as a threshold value. When the sensor has a value over 0.1, color exists, and the sensor is ON. Values vary highly, especially with higher intensities. Therefore, the algorithm cannot be used for accurate color value identification.
The third experiment showed that the proposed approach works well with actual functional inks. However, the algorithm provides sometimes lower values for the sensor as in experiment two. This might be from the label background color differences or calibration of the ink intensity. In the third experiment, all CIEDE2000 parameter combinations provided values that ranged quite widely. In practice, this means that the approach is suitable for sensor state recognition. However, it fails when used to recognize the absolute value of the sensor.
Past research has shown that it is possible to detect colors with smartphone cameras [6]. However, due to restrictions like camera accuracy, ambient light, print, and the paper quality, limited amount of color shades can be recognized [2, 22]. In the past, multiple approaches for color identification in barcodes have been proposed, for example, by John and Raahemifar [20], Bagherinia and Manduchi [2]. In these approaches, different colors represent different data. The same restrictions apply when functional ink operates as a sensor, and a limited amount of shades can be recognized. The most reliable way to use functional ink is to use high enough color intensity. It is challenging to recognize the actual shade of the functional ink. However, recognition of functional inks states, ON or OFF, is quite straightforward. The proposed approach cannot recognize shades accurately, as there is variation in results. One possible approach to overcome this problem might be calibration. In traditional calibration, color charts are used for camera calibration [2]. However, in the case of smart tags, calibration has a small amount of data in its use. These data include only labels background color (paper color), black cells of the marker, and possible color reference points. Together with calibration, reference points can be used. With reference points, it is possible to compare the state of functional ink into specified reference values [6]. However, if reference points are used, they must be printed with color as close as possible to functional ink’s value in a specific state.
Actual color shade recognition with functional ink is a challenging topic as the shade is depending on multiple factors. Ink intensity is the most important factor. However, the direction of state change that occurred last has its impact. In other words, color is slightly different if it has been reached during a negative or positive change. Also, change between states has different speeds in different environments and colors. In some cases, when changing from color to transparent, some color can remain [24].
Conclusions
To support the data-driven applications in the FMCG sector, more data need to be collected. Without this data collection, it is not possible to implement optimal decision-making processes. Currently, the obstacle is not on cloud services or data processing. But on how we collect data from low-cost items and track & trace them during their life-cycle. One approach is to integrate low-cost smart tags with functional ink into items. Functional ink would be able to react to environmental changes and work as a sensor.
Implementation of such markers can be done in various ways, and this paper presented a low-cost way to embed functional ink inside a standard marker. With this approach, the marker has two features: a unique identifier and sensor. Unique identifiers are used to track the item, and the sensor can react to environmental changes. Different color and ink combinations can react to different environmental variables like temperature, time, and humidity. Information from the sensor can be decoded by observing the color changes. One approach to recognize the color and state of the sensor is to use a simple color comparison. CIEDE2000 algorithm with (2.76,1.58,1) parametric values fits for this purpose, and it works reliably when the sensor’s color intensity is 40% or higher. On lower intensities, proposed approach sometimes provides incorrect results.
Yet, more work is needed in the field of color recognition. If color can be recognized accurately, it would allow more use-cases around functional inks and make functional inks usable for use in challenging environments.
Future Work
More research is needed on the computer vision side, especially on how to decode sensor color and its value better accuracy. Also wrap** and challenging conditions might impact proposed approach. Some options, which might achieve better results, could be advanced mathematical calculations, usage of reference colors, or using calibration methods. Usage of machine learning (ML) or artificial (AI) intelligence could also be one option. However, this might make the solution more complex, unless ML/AI solution can be run in a decoding device, rather than in the cloud. When ML/AI is considered, it could be possible to build a rich database containing information about data to color conversion or artificial intelligence models can be trained and used [33].
References
Abad E, Palacio F, Nuin M, De Zarate AG, Juarros A, Gómez JM, Marco S (2009) Rfid smart tag for traceability and cold chain monitoring of foods: demonstration in an intercontinental fresh fish logistic chain. J Food Eng 93(4):394–399
Bagherinia H, Manduchi R (2011) A theory of color barcodes. In: 2011 IEEE International Conference on Computer Vision Workshops (ICCV Workshops), pp 806–813. https://doi.org/10.1109/ICCVW.2011.6130335
Bilgin M, Backhaus J (2017) Intelligent codes by controlled responsiveness to external stimuli. In: Printing future days 2017 7th international scientific conference on print and media technology, pp 85–90
Bulan O, Monga V, Sharma G (2009) High capacity color barcodes using dot orientation and color separability. In: Delp EJ III, Dittmann J, Memon ND, Wong PW (eds) Media forensics and security, vol 7254. International Society for Optics and Photonics, SPIE, Bellingham, pp 397–403
Bulan O, Sharma G (2011) High capacity color barcodes: per channel data encoding via orientation modulation in elliptical dot arrays. IEEE Trans Image Process 20(5):1337–1350
Chen Y, Fu G, Zilberman Y, Ruan W, Ameri SK, Zhang YS, Miller E, Sonkusale SR (2017) Low cost smart phone diagnostics for food using paper-based colorimetric sensor arrays. Food Control 82:227–232
Farizal, Abhirama P (2019) Feasibility analysis of implementing logistic information system with internet of things technology on a fmcg company. In: 2019 IEEE 6th international conference on engineering technologies and applied sciences (ICETAS), pp 1–6. https://doi.org/10.1109/ICETAS48360.2019.9117563
Fuertes G, Soto I, Carrasco R, Vargas M, Sabattin J, Lagos C (2016) Intelligent packaging systems: sensors and nanosensors to monitor food quality and safety. J Sens 2016:4046061
Gao T, Tian Y, Zhu Z, Sun D (2020) Modelling, responses and applications of time-temperature indicators (TTIs) in monitoring fresh food quality. Trends Food Sci Technol 99:311–322
Ghinea R, Herrera L, Ionescu A, Pomares H, Pulgar R, Paravina R et al (2011) Dental ceramics: a ciede2000 acceptability thresholds for lightness, chroma and hue differences. J Dent 39:e37-44
Ghinea R, Pérez MM, Herrera LJ, Rivas MJ, Yebra A, Paravina RD (2010) Color difference thresholds in dental ceramics. J Dent 38:e57–e64
Gligoric N, Krco S, Hakola L, Vehmas K, De S, Moessner K, Jansson K, Polenz I, van Kranenburg R (2019) Smarttags: Iot product passport for circular economy based on printed sensors and unique item-level identifiers. Sensors 19(3):586
Grillo A, Lentini A, Querini M, Italiano GF (2010) High capacity colored two dimensional codes. In: Proceedings of the international multiconference on computer science and information technology, pp 709–716. https://doi.org/10.1109/IMCSIT.2010.5679869
Hakola L, Vehmas K (2018) Functional ink formulation for individualized smart tags. In: NIP & digital fabrication conference, number 1. society for imaging science and technology, pp 211–214. https://doi.org/10.2352/ISSN.2169-4451.2018.34.211
Harvey J (2007) Mechanical engineers’ handbook: materials and mechanical design, volume 1, vol 1. Wiley, Hoboken, pp 1423–1436
International Organization for Standardization (2011) Automatic identification and data capture techniques—bar code symbol print quality test specification - two-dimensional symbols. Standard, International Organization for Standardization, Geneva
International Organization for Standardization (2015) Automatic identification and data capture techniques—qr-code bar code symbology specification. Standard, International Organization for Standardization, Geneva
International Organization for Standardization (2018) Information technology—international symbology specification—data matrix. Technical report
Isohanni J (2021) Qr-code sensor decoder. https://github.com/jariisohanni/QR-code-sensor-decoder. Accessed 5 Aug 2021
John RA, Raahemifar K (2015) Designing a 2d color barcode. In 2015 IEEE 28th Canadian Conference on electrical and computer engineering (CCECE), pp 297–301. https://doi.org/10.1109/CCECE.2015.7129203
Kalpana S, Priyadarshini SR, Leena MM, Moses JA, Anandharamakrishnan C (2019) Intelligent packaging: trends and applications in food systems. Trends Food Sci Technol 93:145–157
Kim M, Song K, Kang M (2018) No-reference contrast measurement for color images based on visual stimulus. IEEE Access 6:1
Kirwan M (2012) Handbook of paper and paperboard packaging technology, 2nd edn. Wiley, Hoboken
Kulčar R, Friškovec M, Knešaurek N, Sušin B, Klanjšek Gunde M (2009) Colour changes of uv-curable thermochromic inks. In: Proceedings of the 36th international research conference of iarigai advanced in printing and media technology, pp 429–434
Hallcrest LCR (2021) Thermochromic free flowing powder technical data. Whitepaper, LCR Hallcrest
Li L, Qiu J, Lu J, Chang C (2016) An aesthetic qr-code solution based on error correction mechanism. In Journal of Systems and Software. 116:85–94. https://doi.org/10.1016/j.jss.2015.07.009
Li Z, Chen W (2014) D&S Barcode: A Dynamic and Sensitive Barcode for Intelligent Environment Monitoring. In: 2014 International Conference on Intelligent Environments, pp 47–51. https://doi.org/10.1109/IE.2014.14
Lindqvist U, Eiroma K, Hakola L, Jussila S, Kaljunen T, Moilanen P, Rusko E, Siivonen T, Välkkynen P (2008) Technical innovations and business from printed functionality. Number 2436 in VTT Tiedotteita—Meddelanden—Research Notes. VTT Technical Research Centre of Finland, Finland. Project code: 4312
Liu H, Huang M, Liu Y, Wu B, Xu Y (2012) Color difference evaluation and calculation for digital and printed images. In: NIP & Digital Fabrication Conference 2012(1):140–143
López A, Guzmán GA, Di Sarli AR (2016) Color stability in mortars and concretes. Part 1: study on architectural mortars. Constr Build Mater 120:617–622
Luo MR, Cui G, Rigg B (2001) The development of the cie 2000 colour-difference formula: Ciede 2000. Color Res Appl 26(5):340–350
Mangine H, Jakes K, Noel C (2005) A preliminary comparison of cie color differences to textile color acceptability using average observers. Color Res Appl 30(4):288–294
Mercan ÖB, Kılıç V, Şen M (2021) Machine learning-based colorimetric determination of glucose in artificial saliva with different reagents using a smartphone coupled \(\mu\)PAD. Sens Actuators B Chem 329:129037
Mohebi E, Marquez L (2015) Intelligent packaging in meat industry: an overview of existing solutions. J Food Sci Technol 52(7):3947–3964
Nguyen C, Vo V, Cong Ha N (2022) Develo** a computer vision system for real-time color measurement—A case study with color characterization of roasted rice. J Food Eng 316:110821
Parikh D, Jancke G (2008) Localization and segmentation of a 2d high capacity color barcode. In: 2008 IEEE workshop on applications of computer vision, pp 1–6. https://doi.org/10.1109/WACV.2008.4544033
Pecho OE, Ghinea R, Alessandretti R, Pérez M, Bona AD (2016) Visual and instrumental shade matching using CIELAB and CIEDE2000 color difference formulas. Dent Mater 32(1):82–92
Plimmer J (2013) Augmenting and securing the consumer brand experience through smart and intelligent packaging for food, beverages and other fast-moving consumer goods, In: Trends in packaging of food, beverages and other fast-moving consumer goods (FMCG), pp 35–57. https://doi.org/10.1533/9780857098979.35
Poyatos-Racionero E, Ros-Lis JV, Vivancos J, Martínez-Máñez R (2018) Recent advances on intelligent packaging as tools to reduce food waste. J Clean Prod 172:3398–3409
Qian J, Du X, Zhang B, Fan B, Yang X (2017) Optimization of QR-Code readability in movement state using response surface methodology for implementing continuous chain traceability. Comput Electron Agric 139:56–64
Querini M, Italiano G (2014) Reliability and data density in high capacity color barcodes. Comput Sci Inf Syst 11:1595–1615
Ramalho J, Correia S, Fu L, Dias L, Adão P, Mateus P, Ferreira R, André PS (2020) Super modules-based active qr-codes for smart trackability and iot: a responsive-banknotes case study. npj Flex Electron 4(1):1–9
Realini CE, Marcos B (2014) Active and intelligent packaging systems for a modern society. Meat Sci 98(3):404–419
Robertson G (2016) Food packaging: principles and practice, 3rd edn. CRC Press, Boca Roton
Salmerón JF, Rivadeneyra A, Agudo-Acemel M, Capitán-Vallvey LF, Banqueri J, Carvajal MA, Palma AJ (2014) Printed single-chip uhf passive radio frequency identification tags with sensing capability. Sens Actuators A Phys 220:281–289
Schmid P, Welle F (2020) Chemical migration from beverage packaging materials—a review. Beverages 6(2):37–55
Schmidt L, Mitton N, Simplot-Ryl D (2009) Towards unified tag data translation for the internet of things. In: 2009 1st International Conference on wireless communication, vehicular technology, information theory and aerospace electronic systems technology, pp 332–335. https://doi.org/10.1109/WIRELESSVITAE.2009.5172469
SICPA Securink Corp (2016) Thermochromic inks. Whitepaper, SICPA Securink Corp
Stanislav B, Igor M, Kristijan G (2015) Packaging printing today. In: Faculty of Graphic Arts, University of Zagreb, Croatia Packaging Printing Today, acta graphical, vol 26(4). pp 27–33, SSN: 1848-3828
Subpratatsavee P, Kuacharoen P (2012) An implementation of a high capacity 2D barcode. In: Papasratorn B, Charoenkitkarn N, Lavangnananda K, Chutimaskul W, Vanijja V (eds) Advances in information technology. Springer Berlin Heidelberg, Berlin, pp 159–169
TagItSmart (2017) Initial enablers for smarttags. Technical report
Tao M, Ma X, Huang X, Liu C, Deng R, Liang K, Qi L (2020) Smartphone-based detection of leaf color levels in rice plants. Comput Electron Agric 173:105431
Tarjan L, Šenk I, Tegeltija S, Stankovski S, Ostojic S (2014) A readability analysis for QR-Code application in a traceability system. Comput Electron Agric 109:1–11
Taveerad N, Vongpradhip S (2015) Development of color qr-code for increasing capacity. In: 2015 11th International Conference on signal-image technology internet-based systems (SITIS), pp 645–648. https://doi.org/10.1109/SITIS.2015.42
Thiesse F, Michahelles F (2006) An overview of epc technology. Sens Rev 26(2):101–105
Tiwari S (2016) An Introduction to QR-Code Technology. 2016 International Conference on Information Technology (ICIT) 322(10):39–44
Triebel D, Reichert W, Bosert S, Feulner M, Okach DO, Slimani A, Rambold G (2018) A generic workflow for effective sampling of environmental vouchers with UUID assignment and image processing. Database J Biol Databases Curation 2018
Đurđević S, Novaković D, Zeljković Ž, Avramović D (2016) Using augmented reality technology for controlling state of smart packaging products. Preliminary Report, pp 427–437
Wasule S, Metkar S (2017) Improvement in two-dimensional barcode. Sādhanā 42(7):1025–1035
Yam KL (2012) 8—Intelligent packaging to enhance food safety and quality. In: Yam KL, Lee DS (eds) Emerging food packaging technologies. Woodhead Publishing Series in Food Science, Technology and Nutrition. Woodhead Publishing, Sawston, pp 137–152
Zabala S, Castán J, Martínez C (2015) Development of a time-temperature indicator (TTI) label by rotary printing technologies. Food Control 50:57–64
Zhu D, Beeby SP, Tudor MJ, Harris NR (2011) A credit card sized self powered smart sensor node. Sens Actuators A Phys 169(2):317–325
Funding
Open access funding provided by Abo Akademi University (ABO). This work was supported by Finnish Cultural Foundation’s Central Ostrobothnia Regional Fund (Grant Number 25211242).
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Competing interests
The authors have not disclosed any competing interests.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
Isohanni, J. Use of Functional Ink in a Smart Tag for Fast-Moving Consumer Goods Industry. J Package Technol Res 6, 187–198 (2022). https://doi.org/10.1007/s41783-022-00137-4
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s41783-022-00137-4