Keywords

1 Introduction

Online marketplaces, locations on Internet where people can purchase and sale services and goods, have highly increased in the last couple of decades. Recently, various marketplaces for exchanging AI models have been introduced [1]. In these marketplaces the AI models and machine learning (ML) algorithms have been monetized and offered as products.

AWS MarketplaceFootnote 1 by Amazon enables its customers to find a large variety of pre-built models and algorithms covering a wide range of use cases and domains related to Business Analytics, Computer Vision, Healthcare, and Text and Language Processing. A subscription-based model with charging per hour, day, etc. is primarily adopted. A monthly based subscription model is also offered by Akira.AI.Footnote 2 This marketplace offers AI models especially for solutions related to Text Analysis and Computer Vision along with access to processing, storage, and network resources for enabling the execution of AI models. Pretrained models for a wide variety of sectors are also available at the Gravity AIFootnote 3 and Modelplace AIFootnote 4 (specialized in Computer Vision) marketplaces. The latter enables the real-time execution of models through web–browser interfaces. Other marketplacesFootnote 5 move a step further from live execution to shared building of models among developers by providing software development kits (SDKs).

Some specific AI marketplaces for healthcare domain are also available such as the Imaging AI MarketplaceFootnote 6 by IBM. It is a centralized marketplace that enables healthcare providers to discover, purchase, and manage applications that provide the latest AI-powered tools. In this marketplace, researchers and developers can reach a large community of customers for their specific AI applications for eHealth domain and take advantage of the provided infrastructure and deployment processes. To this direction, Nuance CommunicationsFootnote 7 has introduced its marketplace for Healthcare AI solutions as well providing similar functionalities as the IBM one.

In addition to word-leaders’ approaches and market-ready solutions, EC-funded research projects have presented various marketplaces for listing or even trading solutions including AI/ML algorithms. AI4EUROPE or AI on Demand [2] offers a trustworthy open-source platform for the development, training, and sharing of AI/ML models. However it is considered more as an open code repository as it lacks business logic in comparison with commercial marketplaces. MARKET4.0 project [3] develops a multi-sided business platform for plug and produce industrial product service systems. The European Factory Foundation and EFPF project [4] offers a Portal/Marketplace as part of its interoperable Data Spine that includes solutions coming from previous EU projects and third-parties’ initiatives. However the solutions are a mix of software solutions, products, and services. Other marketplaces related to manufacturing domain and Industry 4.0 were provided by projects like v-fos [5] (which offers an application marketplace with an embedded SDK) and NIMBLE [6] that has introduced a federated interoperable eco-system for B2B connectivity. Some other approaches [7] coming from research introduced AI as enablers for marketplaces by combining virtual agents with semantics [8, 9] for automated negotiations in manufacturing marketplaces [10]. In Boost 4.0 project a common European Data Space for manufacturing was introduced instead of a Marketplace. However, it contains AI services connected to available data sources based on IDSAFootnote 8 architecture. The latter supports also the establishment of AI MarketplaceFootnote 9 that is a meeting place for AI providers and consumers. Recently, PoP-Machina projectFootnote 10 proposed a collaboration platform for makers that provides a marketplace based on a blockchain network infrastructure. However, it is focused more on collaborative design and not on AI model exchange. A blockchain approach for building marketplaces for AI was also introduced by IBM [11]. It was a back-end implementation regarding trusted transactions and not a full operational marketplace.

As it is perceived there are market-ready solutions in the field of AI marketplaces, however they are focused on cases related to health, text recognition, computer vision, etc. and not to manufacturing and Industry 4.0/5.0 domains. On the contrary, there are marketplaces coming from research field that are related to Industry 4.0/5.0 domain, but either they lack some business logic or they collect heterogeneous solutions and even physical products, so they cannot be considered as marketplaces for exchanging of AI/ML models. In the current work, we are introducing the knowlEdge project’s [12, 13] Marketplace that aims to deliver a marketplace for exchanging AI models for smart manufacturing using blockchain services and smart contracts as the core of its business logic. The introduced marketplace can act as an enabler for intelligent production as it collects and offers AI solutions related to manufacturing domain able to solve various kinds of problems in factories.

Following this introductory section, the next section presents the knowlEdge AI Marketplace’s main functionalities and its high-level architecture. Section 3 presents the core technical parts and interfaces of the knowlEdge Marketplace. The conclusions are drawn in Sect. 4.

2 Functionalities and Proposed System Architecture of knowlEdge Marketplace

The introduced marketplace for AI models regarding smart manufacturing offers a series of functionalities common to normal marketplaces and stores for services and products. In particular, it offers:

  • A user-friendly web-based interface to enable trading of AI algorithms and models

  • Trusted trades among the stakeholders, protection of Intellectual Property Rights (IPR) and security

  • Profiles and role management functionalities

  • Search functionalities based on various features

  • Reviews and ratings regarding users and AI models

To support the abovementioned functionalities, the knowlEdge Marketplace incorporates a series of technologies and components in its architecture as it is depicted in Fig. 1. They are distinguished in three main categories: the back-end part related to AI models description and management, the back-end part related to business logic and transactions, and the front-end part related to user-centric services such as interfaces.

Fig. 1
An illustration of a high-level architecture of the knowledge A I model Marketplace. The backend part is related to A I model description and management and business logic and transactions. The front-end part is related to user-centric services.

High-level architecture of the knowlEdge AI model Marketplace

Full size image

In particular, the User-Centric Services module provides a series of functionalities related to user experience such as UIs, search functionalities, user profile management, etc.

NFT-Based Monetization Framework for AI Models based on Smart Contracts provides all the functionalities related to business logic based on blockchain. It offers smart contracts (or chaincode) for fungible and non-fungible Tokens (FTs and NFTs), marketplace, and mint notary. Furthermore, secure access services are also part of this module.

AI Model Repository is responsible for the modeling of AI/ML algorithms, their storage, and the provision of management services such as CRUD (Create, Read, Update, and Delete) operations.

Various APIs have also been developed to enable the different module communication based on HTTP protocol.

All the core modules of the introduced marketplace are presented in the following section.

3 Implementation of knowlEdge Marketplace for Exchanging AI Models in Industry 5.0

In this section, the core modules of the knowlEdge Marketplace are described. We start with the way, the data is stored in the knowlEdge AI Model Repository (Sect. 3.1), before we discuss the Monetization Framework (Sect. 3.2) and conclude it with the User Interfaces (Sect. 3.3).

3.1 knowlEdge AI Model Repository

The knowlEdge AI Model Repository is a central cloud-hosted component, which manages a database of AI models and their corresponding meta-information. For these purposes, it consists of four main components: the knowlEdge Repository Management, the Model Database, the Metadata Database, and the Historical Data Store (see Fig. 2). In general, the AI Model Repository provides all the necessary functionality to Marketplace regarding the modeling of the AI and the management of data and metadata in the Marketplace that are related to AI models. So, besides the four aforementioned components, an Ontology that enables the modeling of Marketplace’s metadata is a core part of the Repository as well.

Fig. 2
An internal architecture of the knowledge A I Model Repository. Four main components are the knowledge Repository Management, the Model Database, the Metadata Database, and the Historical Data Store.

The internal architecture of the knowlEdge AI Model Repository

Full size image

3.1.1 Overview of Key Components

The knowlEdge Repository Management is the central component connecting the other components and offering services over a REST API, which follows the OpenAPI [14] specification and is the single interface of the knowlEdge Repository. It offers a feature-rich possibility to query for AI models and datasets to identify similarities between different problems and solutions.

The Metadata Database is a MongoDB [15], which stores the metadata of the knowlEdge Repository. This metadata follows the knowlEdge Ontology (see Sect. 3.1.2) to ensure a high level of usable information for all datasets and models present in the repository.

The Model Database is used to store the actual model specification files in a Hadoop [16] Distributed File System. The model files can be presented in ONNXFootnote 11 or PMML [17] format to make sure that as many different models as possible can be described for the knowlEdge Repository.

The Historical Data Store is used to store the datasets a model is trained on. With these datasets also present, it is possible to benchmark new models on the same datasets and directly compare the performance of models.

3.1.2 Ontology

The knowlEdge Ontology has been developed to ensure that a wide variety of metadata are available for the knowlEdge repository. The Ontology consists of 12 different types of entities (see Fig. 3). In this entity–relationship diagram, the most important entities and relationships of the knowlEdge Ontology are given and show the possibilities of the hierarchical structure of this Ontology. It shows the technical ways the knowlEdge Repository stores the data according to it. These entities can be split into user-related (User), model-related (Model, IO Vector, Model Specification and Model Type), data-related (Task, Analysis Type, Application, Data, Property and Property Type), and performance-related (Performance Evaluation) Entities. The main entities/classes are as follows:

  • User: A User is identified by its unique name and email address. Furthermore, the timestamp of the time it was created is stored. A User can be owner of several Models and creator of several Application, Data, and Property Type entities.

  • Model: A Model contains its name, a description, and the timestamp of its creation as well as a link to a Model Specification and the Model Type it instantiates. It can have hierarchical children and a parent as well as multiple IO Vectors as input and output.

  • IO Vector: An IO Vector contains its name, dimensionality, and data type, which can be of type integer, float, Boolean, or categorical. It can be input and output to several Models.

  • Model Specification: A Model Specification contains the actual model file, i.e., an ONNX or PMML file describing the model.

  • Model Type: A Model Type has its name. It can be child to another Model Type and parent to several Model Types. It can be instantiated by several Models.

  • Task: A Task consists of its name, the timestamp of its creation, its Analysis Type, Application, and Data. It is created by a single User, can have multiple Tasks as children, and can have several Models trained for it. It may be child of another Task.

  • Analysis Type: An Analysis Type has a name and the timestamp of its creation. It is part of a Task and can be child to one and parent to many Analysis Types.

  • Application: An Application has a name, a description, and the timestamp of its creation. It is part of a Task, created by a User, and can be child to one and parent to many Applications.

  • Data: A Data entity consists of its name and description, the Task it is part of, and the timespan it was gathered during. It is created by a User, consists of several Properties, and may inherit from one and be parent for several Data entities.

  • Property: A Property consist of its name, the Property Type it instantiates, and the Data entity it belongs to.

  • Property Type: A Property Type consists of its name, creation time, and type, which may be Boolean, integer, float, or categorical. It is created by a User and may be instantiated by several Properties. A Property Type can be based on one Property Type, and there can be multiple Property Types based on one Property Type.

  • Performance Evaluation: A Performance Evaluation represents the actual performance of an AI Model on a Task. It is linked to both these entities and contains the performance measurement as well as the information which measure this represents.

The model specifications in the Model Database will be stored in the ONNX or PMML Format. These two formats offer compatibility with a wide range of different machine learning techniques and frameworks to boost interoperability regarding AI models in the proposed marketplace. While PMML focuses on traditional machine learning methods, ONNX is specialized for exchanging deep neural networks. The combination of both formats makes it possible to store a wide range of machine learning models in an easy to use and deploy way.

Fig. 3
A structural representation of the knowlEdge Ontology. It comprises 12 different types of entities. The main entities are user, model, I O vector, model specification, model type, task, analysis type, application, data, property, property type, and performance evaluation.

The technical structure of the knowlEdge Ontology

Full size image

3.2 NFT-Based Monetization Framework for AI Models

Besides the Ontology for describing the Marketplace metadata and the repository services for the management of AI models, a set of services regarding the monetization-, security-, and business-related concepts was required for the delivery of the proposed AI Model Marketplace.

Therefore, in the context of the knowlEdge Marketplace, a number of blockchain-based services have been developed and deployed. An end-to-end decentralized AI Model Marketplace that enables the monetization of AI models while ensuring security, auditability, and verifiability has been implemented based on blockchain technology. To guarantee ownership of AI models, each model is treated as a unique asset on the distributed ledger, represented as non-fungible tokens (NFTs). The use of NFTs provides additional functionalities, including ownership transfer. The marketplace is based on the presupposition that participants share a common value system, and fungible tokens are used as the equivalent of real-world fiat currency.

In Table 1, we provide the various actor roles and the means under which they engage with the marketplace platform, i.e., their capabilities.

Table 1 Roles and capabilities of users in the system
Full size table

Note that the terms “AI Model Producer,” “AI Model Researcher,” and “AI Model Developer” are used interchangeably to refer to the same actor. Similarly, the terms “AI Model Consumer” and “Marketplace Customer” refer to the same actor. Lastly, note that the same real-world entity can potentially enact in all of the aforementioned roles, e.g., an AI model producer can also act as a consumer, or marketplace customer for AI models produced by others.

The following functionalities are available in the DLT-based AI Model Marketplace, building on the core set of functionalities that were previously outlined. The functionalities are as follows:

  • AI model producers, represented as NFT owners, can advertise their willingness to sell access to individual AI model binary files at a price of their choice.

  • Each AI model producer can query the AI models they have advertised on the marketplace.

  • Each AI model producer can retract advertised AI models at any time.

  • Any entity can query all AI models advertised on the marketplace.

  • Interested AI model consumers can purchase access to any advertised AI model, provided they have sufficient coin balance.

  • AI model consumers retain access rights to purchased AI model binary files, even if the models are later retracted from the marketplace.

  • Each consumer can query the marketplace for a list of all successful purchases.

  • External entities, such as an AI Model repository, can securely verify that an actor requesting access to an AI model binary file is a legitimate consumer who has previously performed a successful purchase.

A detailed diagram of the architecture can be found in Fig. 4. The diagram depicts the involved users, the components of the blockchain infrastructure, and intuitive descriptions of their interactions.

Fig. 4
An architecture diagram of the A I model monetization framework. It presents the involved users, the components of the blockchain infrastructure, and intuitive descriptions of their interactions.

High-level overview of the AI model monetization framework’s architecture

Full size image

There are also several off-chain components that play different roles. For example, we need to account for the physical storage of AI model files and provide an identity and access management infrastructure for the actors and the services they consume. Additionally, we need to include various integration and deployment-related components such as API gateways and dashboards (user interfaces) for the involved actors. The list of components along with a succinct description of their function can be found below:

  • Hyperledger Fabric (henceforth, HLF) Community Management (CM) API: A federated identity authorization service that, apart from supporting all standard OAuth 2.0 and OpenID Connect flows, encompasses, as part of the user registration (onboarding) process, private key and X.509 digital certificate generation, which are subsequently stored in an HLF-compliant secure wallet store.

  • HLF Wallet: Implementation of HLF’s Wallet interface employing MongoDB as a storage medium for Hyperledger Fabric’s Go SDK. This is employed internally by the HLF SC Gateway component (see below).

  • HLF Smart Contract (SC) Gateway: a configurable microservice that exposes functions of any smart contract deployed on any arbitrary Hyperledger Fabric channel as HTTP endpoints

  • NFT Metadata Store: a REST API that exposes endpoints for storing and retrieving metadata files associated with NFTs

  • NFT Chaincode: the smart contract that implements the entire non-fungible token-related functionality

  • FT Chaincode: the smart contract that implements the entire fungible token-related functionality

3.3 User Interfaces and Functionalities

Besides the core back-end services and the corresponding modules that were presented in the previous section, a module focused on the delivery of front-end-related services is also included in knowlEdge Marketplace. This module does not only include interfaces but also supports some user-related functionalities such as search capabilities and user’s profile management. They are considered in the same building block as they are strictly connected to a front-end theme that was used and their functionality is derived from them.

Regarding the interfaces design, best practices were used. The design pillars that were followed were the esthetic and minimalistic design, the use of common and consistent UI elements, the adoption of widely used input controls and navigational elements, the error prevention and good error messaging, etc. The UIs were implemented as web-based interfaces using technologies such as Angular, Bootstrap, and Nebular. The ngx-admin template was used to enable faster implementation as it is a popular admin dashboard based on Angular and it is free and open source. It is efficient as it is packed with a huge number of UI components and it is highly customizable.

The UIs enable the exploration of various available AI models in different views (see Fig. 5) based on the user’s preferences (grid view and list view). The search functionalities provide various filters such as AI models owner, category of the algorithm, price range, rating, etc. Text-based search is also supported, so the user can type text related to the model’s name, keywords, and other metadata.

Fig. 5
A screenshot titled knowlEdge Marketplace. It provides the list of A I models. From the Homepage, Marketplace is selected. Search results give R N N models 4, 3, and 2.

List of AI models

Full size image

By selecting a model, the user is able to read details (see Fig. 6) such as description, specifications of the model, and metadata such as rating, price, and owner. Furthermore, any datasets connected to a model are also visible. All the data available to UI are dynamically retrieved from Repository and Monetization modules. The user can also select to add to cart a model in order to purchase it based on the NFT monetization module.

Fig. 6
A screenshot titled knowlEdge Marketplace. It provides details of the A I model. From the Homepage, Marketplace is selected. R N N model 4 description and specification are provided.

AI model details

Full size image

Besides exploring and purchasing AI models, a user can act as a provider and deploy his/her own AI model by using corresponding interfaces (Fig. 7) that are available in a kind of a step wizard form. First, the user adds the dataset details that were used for training a model. Then the general details regarding the task/application that the model is related to (e.g., predictive maintenance) are added. After that, the user adds AI model details such as the type, input and output, model format, and connections with other models and deploys the model itself (e.g., an ONNX file, etc.).

Fig. 7
A screenshot titled knowlEdge Marketplace. It uploads the A I model. From My Repository, the New Model is selected. R N N model 4 dataset details, task details, and model details are provided.

Uploading an AI model

Full size image

4 Conclusions

The design and the implementation of a marketplace for exchanging AI models related to Industry 5.0 and smart manufacturing are introduced. The knowlEdge Marketplace highlights the main components that a marketplace for AI models should include. A component to enable the modeling of AI models/algorithms and their metadata based on standards has been defined as a necessity. Moreover, AI developers should be able to provide their models based on widely used formats and standards in such marketplaces. Furthermore, services to enable trusted transactions and sharing, along with security and protection of ownership in AI marketplaces, have been found as core concepts that should be covered as well. The use of blockchain technology for this kind of services has been proved as an ideal candidate as it provides all the necessary concepts regarding monetization and secure and trusted transactions. Moreover, user-friendly and easy-to-use interfaces were another important factor that should be considered as in the end, as any other marketplace, it is focused to end-users.

Regarding the next steps, the knowlEdge Marketplace focuses on further testing and evaluation by domain experts targeting to final improvements so to be considered as a completely market-ready solution. The plan is for this marketplace to be one of the core AI marketplaces in Europe related to AI models exchanged in Industry 5.0 era.