Improving data reliability assessment in ETL processes through quality scoring technique in data analytics
The foundation of a relevant and accurate data analysis is reliable data. Technique and measurement are essential to evaluate current data quality regarding reliability and establish a baseline for ongoing improvement initiatives. Without tools or visualizations, data engineers may find it challengi...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Politeknik Negeri Padang
2024
|
| Online Access: | http://psasir.upm.edu.my/id/eprint/117927/ http://psasir.upm.edu.my/id/eprint/117927/1/117927.pdf |
| _version_ | 1848867382368403456 |
|---|---|
| author | Atika Razali, Nor Famiera Baharom, Salmi Abdullah, Salfarina Admodisastro, Novia Indriaty |
| author_facet | Atika Razali, Nor Famiera Baharom, Salmi Abdullah, Salfarina Admodisastro, Novia Indriaty |
| author_sort | Atika Razali, Nor Famiera |
| building | UPM Institutional Repository |
| collection | Online Access |
| description | The foundation of a relevant and accurate data analysis is reliable data. Technique and measurement are essential to evaluate current data quality regarding reliability and establish a baseline for ongoing improvement initiatives. Without tools or visualizations, data engineers may find it challenging to monitor and maintain the reliability of the massive data from the extraction, transformation, and loading (ETL) data load process. Data reliability assessment is a helpful technique in analyzing the quality of data reliability and information on the present state of data before commencing any analytics. The proposed technique hinges on the metric and measurement defining data reliability and the dashboard platform where the integration with the user in dictating the weight of data and the final output, which is the final data reliability score, will be projected. The score obtained affirms whether improvements are needed on the data or if an organization can proceed with data analytics. The technique considers the data extraction, transformation, and loading (ETL) procedures used to gather datasets. Data significance or weight was determined according to the analytics needs and preferences, indicating an acceptable score for generating insights. Ultimately, when utilizing the data reliability assessment metrics technique, we are credited with an overall picture of our data’s reliability aspect, as only one look is offered based on the intended analysis. This new approach boosts the confidence among data practitioners and stakeholders, especially those relying on findings generated from data analysis. Furthermore, the overview assists in enhancing the current state of data, where the derived score helps identify possible areas of improvement in the ETL process. Accuracy and efficiency assessment of the proposed technique also showed positive feedback in measuring the method in measuring the reliability of data. |
| first_indexed | 2025-11-15T14:35:36Z |
| format | Article |
| id | upm-117927 |
| institution | Universiti Putra Malaysia |
| institution_category | Local University |
| language | English |
| last_indexed | 2025-11-15T14:35:36Z |
| publishDate | 2024 |
| publisher | Politeknik Negeri Padang |
| recordtype | eprints |
| repository_type | Digital Repository |
| spelling | upm-1179272025-06-17T02:53:59Z http://psasir.upm.edu.my/id/eprint/117927/ Improving data reliability assessment in ETL processes through quality scoring technique in data analytics Atika Razali, Nor Famiera Baharom, Salmi Abdullah, Salfarina Admodisastro, Novia Indriaty The foundation of a relevant and accurate data analysis is reliable data. Technique and measurement are essential to evaluate current data quality regarding reliability and establish a baseline for ongoing improvement initiatives. Without tools or visualizations, data engineers may find it challenging to monitor and maintain the reliability of the massive data from the extraction, transformation, and loading (ETL) data load process. Data reliability assessment is a helpful technique in analyzing the quality of data reliability and information on the present state of data before commencing any analytics. The proposed technique hinges on the metric and measurement defining data reliability and the dashboard platform where the integration with the user in dictating the weight of data and the final output, which is the final data reliability score, will be projected. The score obtained affirms whether improvements are needed on the data or if an organization can proceed with data analytics. The technique considers the data extraction, transformation, and loading (ETL) procedures used to gather datasets. Data significance or weight was determined according to the analytics needs and preferences, indicating an acceptable score for generating insights. Ultimately, when utilizing the data reliability assessment metrics technique, we are credited with an overall picture of our data’s reliability aspect, as only one look is offered based on the intended analysis. This new approach boosts the confidence among data practitioners and stakeholders, especially those relying on findings generated from data analysis. Furthermore, the overview assists in enhancing the current state of data, where the derived score helps identify possible areas of improvement in the ETL process. Accuracy and efficiency assessment of the proposed technique also showed positive feedback in measuring the method in measuring the reliability of data. Politeknik Negeri Padang 2024 Article PeerReviewed text en cc_by_sa_4 http://psasir.upm.edu.my/id/eprint/117927/1/117927.pdf Atika Razali, Nor Famiera and Baharom, Salmi and Abdullah, Salfarina and Admodisastro, Novia Indriaty (2024) Improving data reliability assessment in ETL processes through quality scoring technique in data analytics. International Journal on Informatics Visualization, 8 (4). pp. 2195-2202. ISSN 2549-9904 https://joiv.org/index.php/joiv/article/view/3632/1145 10.62527/joiv.8.4.3632 |
| spellingShingle | Atika Razali, Nor Famiera Baharom, Salmi Abdullah, Salfarina Admodisastro, Novia Indriaty Improving data reliability assessment in ETL processes through quality scoring technique in data analytics |
| title | Improving data reliability assessment in ETL processes through quality scoring technique in data analytics |
| title_full | Improving data reliability assessment in ETL processes through quality scoring technique in data analytics |
| title_fullStr | Improving data reliability assessment in ETL processes through quality scoring technique in data analytics |
| title_full_unstemmed | Improving data reliability assessment in ETL processes through quality scoring technique in data analytics |
| title_short | Improving data reliability assessment in ETL processes through quality scoring technique in data analytics |
| title_sort | improving data reliability assessment in etl processes through quality scoring technique in data analytics |
| url | http://psasir.upm.edu.my/id/eprint/117927/ http://psasir.upm.edu.my/id/eprint/117927/ http://psasir.upm.edu.my/id/eprint/117927/ http://psasir.upm.edu.my/id/eprint/117927/1/117927.pdf |