Data Management Concepts

Comparing Paper to Digital Methods

The concept of data management is not new. Over 5,000 years ago clay tokens were used to record specific commodities like numbers of livestock. In cultures without writing systems, a variety of systems were developed to ensure that knowledge was transferred accurately from one generation to the next.

Commonly available articles found on the Internet about managing data tend to fall into two groups. Ones with lots of jargon that leave regular folks in the dark or sales materials focused on a specific product that present an incomplete view of the true work involved in data management. In an effort to make these concepts more clear, this post compares a traditional storage system, physical libraries, with modern digital storage systems.

A Day at the Library

In the days before computers, if you wanted a certain type of information, you could borrow a book from a library. However, you couldn’t go right in and take the book you wanted, you had to follow certain rules. You had to wait until the library was open and often you needed a library card to use the library’s services. Libraries are organized is specific ways, so once in the library, if you knew where to look, you could just go to that section and browse through the materials; however, if you didn’t know where to look you could use the card catalogues to find the book by topic, author or title. If there were no copies of the book you wanted on the shelf, you could go to the reference desk and ask a librarian if the book was available from another library or if a copy was held in reserve in the reference section. When you signed out the book your name and date would be recorded, and often you would be fined if you didn’t return it on time.

Access, Security and Backups

Waiting for the library to open is equivalent to being granted access to a digital resource. Needing a library card is equivalent to needing a user name and password. Similarly, some materials being available on the shelf and others being held in a reference section are comparable to the different levels of access within a data management system. A library containing multiple copies of a book is a good example of on-site backup and copies of the book at different libraries is a great example for off-site backups. Lastly, the process of recording who and when books are signed out is a good example of access and change tracking. All these rules operating together is a good example of the concept of data governance; the rules that determine how data is used and managed.

Models, Storage and Metadata

The way the library’s content is organized is roughly equivalent to the concept of data modelling in digital systems. The physical storage space in the library is equivalent to the type of computer hardware used to store digital files. The climate control systems that protect the books are equivalent to the systems that support that hardware. The card catalogue is the manual equivalent to modern searching tools. The card catalogue also contains information on the book regarding when it was published, who published it and so on; in data management terms this information is called metadata.

Integration, Reporting and Quality Control

In the old days, if you wanted to pull data together from multiple sources you had to do it by hand. If you were lucky, you would find a librarian who had expertise on the subject you were interested in. They would be able to not only point you to the right source materials but also give you an overview of the subject. This is equivalent to a data management system’s reporting dashboard. Similarly the quality of the information in libraries was and remains dependent on the people who select the books and other materials added to the collection. In a similar way, a quality review process is important to include in a modern data collection and storage system to ensure the desired level of quality is obtained.

Technology Doesn’t Replace Planning and Procedures

One thing that often surprises people when transitioning from paper-based data collection and management to digital tools is that policies and procedures become even more important. Putting your files onto a network drive doesn’t magically organize them anymore than taking paper files from boxes and putting them into a shiny new filing cabinet. In the real world transitioning to digital requires a willingness to change, careful documentation and detailed planning. The technology used certainly matters but no technical solution can succeed without the hard work needed to support the change.

Summary

When the Internet was first created, many people thought that libraries would disappear and librarians would be replaced by software. In fact the opposite is true because the basic principles of data management are more relevant than ever as data becomes more and more valuable.

For more practical information on data management practices and recommendations look for future blog posts or Contact Us.