Methodology and evaluation framework
Participants
The evaluation task is a crucial component of our project, which was primarily carried out by ITAINNOVA, in collaboration with the Dig_IT partners and end-users, as one of the main objectives of this task is to evaluate the final deployment of the platform in the different use-case scenarios. Besides, the evaluation of platform’s performance has been defined according to the end-users expectations.
Objective and Outcomes
The purpose of this task is to provide a methodology and evaluation framework to perform a continuous quantification on how the platform is evolving and on the impact of the platform on use-cases’ activities. This evaluation methodology will cover the whole chain of activities from data collection to on-line platform operation.
For such task, two main objectives are defined:
- To define an incremental evaluation procedure that is adapted to each layer of the IIoT platform, generating specific assessment activities for checking and improving the platform services.
- To provide a tool for the control and monitoring of the evaluation procedures.
The use and definition of the evaluation procedure proposed in this task is crucial for the development all other tasks within this work package. For task T6.2 “System Integration”, the use of the proposed evaluation methodology is essential for the assessment of the progress of each WP. In this way, the state of each task is evaluated in order to see if all checkpoints have been fulfilled for the further integration. Besides, the defined KPIs according to the end-users expectations will be used to quantify the performance of the platform in task T6.3 “Testing on real use case scenarios”, where general feedback from all involved stakeholders will be collected, and a best practice guide will be developed with a focus on the scalability of the platform, according to task T6.4 “Comparative analysis, Best practices, Scalability”.
Description
About the work done, an initial version of the evaluation methodology was proposed and shared with all partners, and the evaluation tools were presented (Evaluation Control Sheet, Gantt Charts).
After that, the methodology was improved according to the Project Officer recommendations, and extended to include the KPIs for the proper assessment of the platform’s integration according to the end-users expectations.
Finally, after several follow-up meetings to check the performance and progress of the tools, the final version of the evaluation methodology was presented with the final due dates according to the ones included in the amendment of the Grant Agreement. During the development of this task, three deliverables have been submitted, which correspond to the initial (D6.1), updated (D6.2) and final version (D6.3) of this evaluation procedure. Microsoft Excel was used in order to define evaluation tools developed: the Evaluation Control Sheet, the KPIs and the following-up Gantt Charts.