Technical challenges of providing record linkage services for research
Background: Record linkage techniques are widely used to enable health researchers to gain event based longitudinal information for entire populations. The task of record linkage is increasingly being undertaken by specialised linkage units (SLUs). In addition to the complexity of undertaking probab...
| Main Authors: | , , , , , |
|---|---|
| Format: | Journal Article |
| Published: |
Biomed Central Ltd
2014
|
| Subjects: | |
| Online Access: | http://hdl.handle.net/20.500.11937/37970 |
| _version_ | 1848755193224626176 |
|---|---|
| author | Boyd, James Randall, Sean Ferrante, Anna Bauer, J. Brown, A. Semmens, James |
| author_facet | Boyd, James Randall, Sean Ferrante, Anna Bauer, J. Brown, A. Semmens, James |
| author_sort | Boyd, James |
| building | Curtin Institutional Repository |
| collection | Online Access |
| description | Background: Record linkage techniques are widely used to enable health researchers to gain event based longitudinal information for entire populations. The task of record linkage is increasingly being undertaken by specialised linkage units (SLUs). In addition to the complexity of undertaking probabilistic record linkage, these units face additional technical challenges in providing record linkage ‘as a service’ for research. The extent of this functionality, and approaches to solving these issues, has had little focus in the record linkage literature. Few, if any, of the record linkage packages or systems currently used by SLUs include the full range of functions required. Methods: This paper identifies and discusses some of the functions that are required or undertaken by SLUs in the provision of record linkage services. These include managing routine, on-going linkage; storing and handling changing data; handling different linkage scenarios; accommodating ever increasing datasets. Automated linkage processes are one way of ensuring consistency of results and scalability of service. Results: Alternative solutions to some of these challenges are presented. By maintaining a full history of links, and storing pairwise information, many of the challenges around handling ‘open’ records, and providing automated managed extractions are solved. A number of these solutions were implemented as part of the development of the National Linkage System (NLS) by the Centre for Data Linkage (part of the Population Health Research Network) in Australia.Conclusions: The demand for, and complexity of, linkage services are growing. This presents as a challenge to SLUs as they seek to service the varying needs of dozens of research projects annually. Linkage units need to be both flexible and scalable to meet this demand. It is hoped the solutions presented here can help mitigate these difficulties. |
| first_indexed | 2025-11-14T08:52:25Z |
| format | Journal Article |
| id | curtin-20.500.11937-37970 |
| institution | Curtin University Malaysia |
| institution_category | Local University |
| last_indexed | 2025-11-14T08:52:25Z |
| publishDate | 2014 |
| publisher | Biomed Central Ltd |
| recordtype | eprints |
| repository_type | Digital Repository |
| spelling | curtin-20.500.11937-379702017-09-13T14:10:11Z Technical challenges of providing record linkage services for research Boyd, James Randall, Sean Ferrante, Anna Bauer, J. Brown, A. Semmens, James Medical informatics computing Automatic data processing Medical record linkage Background: Record linkage techniques are widely used to enable health researchers to gain event based longitudinal information for entire populations. The task of record linkage is increasingly being undertaken by specialised linkage units (SLUs). In addition to the complexity of undertaking probabilistic record linkage, these units face additional technical challenges in providing record linkage ‘as a service’ for research. The extent of this functionality, and approaches to solving these issues, has had little focus in the record linkage literature. Few, if any, of the record linkage packages or systems currently used by SLUs include the full range of functions required. Methods: This paper identifies and discusses some of the functions that are required or undertaken by SLUs in the provision of record linkage services. These include managing routine, on-going linkage; storing and handling changing data; handling different linkage scenarios; accommodating ever increasing datasets. Automated linkage processes are one way of ensuring consistency of results and scalability of service. Results: Alternative solutions to some of these challenges are presented. By maintaining a full history of links, and storing pairwise information, many of the challenges around handling ‘open’ records, and providing automated managed extractions are solved. A number of these solutions were implemented as part of the development of the National Linkage System (NLS) by the Centre for Data Linkage (part of the Population Health Research Network) in Australia.Conclusions: The demand for, and complexity of, linkage services are growing. This presents as a challenge to SLUs as they seek to service the varying needs of dozens of research projects annually. Linkage units need to be both flexible and scalable to meet this demand. It is hoped the solutions presented here can help mitigate these difficulties. 2014 Journal Article http://hdl.handle.net/20.500.11937/37970 10.1186/1472-6947-14-23 Biomed Central Ltd fulltext |
| spellingShingle | Medical informatics computing Automatic data processing Medical record linkage Boyd, James Randall, Sean Ferrante, Anna Bauer, J. Brown, A. Semmens, James Technical challenges of providing record linkage services for research |
| title | Technical challenges of providing record linkage services for research |
| title_full | Technical challenges of providing record linkage services for research |
| title_fullStr | Technical challenges of providing record linkage services for research |
| title_full_unstemmed | Technical challenges of providing record linkage services for research |
| title_short | Technical challenges of providing record linkage services for research |
| title_sort | technical challenges of providing record linkage services for research |
| topic | Medical informatics computing Automatic data processing Medical record linkage |
| url | http://hdl.handle.net/20.500.11937/37970 |