Handover Datasets and Data Dictionaries
A borewell handover dataset is the structured record that allows another person to understand and use the project’s observations after the original team has left. It should preserve the observations, their context and their limitations. A polished PDF is useful for communicating conclusions, but it is not a substitute for machine-readable data when future recalculation, trending or comparison is expected.
The data dictionary explains what every field means. It defines units, reference conventions, permitted values, missing-data treatment and relationships between records. USGS guidance describes dictionaries as a means of communicating data structure and meaning, including quality and missing-value codes. The example structure below is a project handover design, not a claim that one universal schema fits every borewell investigation.
Design around the questions the recipient must answer
Identify the expected uses before selecting file formats. An operator may need the current limits and alarm definitions. An engineer may need time series and test configuration. A maintenance team may need equipment identity and intervention history. A data analyst may need field types, units and quality flags. One folder can serve these users if its records are organised by purpose and connected through stable identifiers.
Do not force every kind of information into one oversized spreadsheet. A well register, instrument register, observation table, event log and decision register have different structures. Keeping them separate can improve clarity if the links are documented. The handover should include a simple relationship description showing how a test identifier connects to its well, instruments, observations and issued conclusions.
Define the minimum identity structure
Use stable identifiers for the well, test, instrument and observation or observation series. Document which identifiers are unique and whether they can ever be reused. A row number generated by sorting a spreadsheet is not a dependable observation identity. If records are reordered or filtered, the reference must still point to the same underlying observation.
Keep human-readable names as descriptive fields rather than relying on them as the only keys. Names change and may be duplicated. Record location and construction references appropriate to the project while controlling access to sensitive information. The public article does not need to expose a customer’s precise asset coordinates for the internal dataset to remain technically usable.
Make time unambiguous
Record timestamps with an explicit timezone or offset and state the clock source. Preserve elapsed test time where it is useful, but define its origin and unit. A timestamp and an elapsed-time field should be capable of being checked against each other. Document whether a record represents an instantaneous observation, an interval average or an accumulated total.
When several instruments are involved, record synchronization and any known clock differences. Do not silently shift raw timestamps to make graphs line up. A corrected analysis view may be needed, but preserve the original time values and the correction rule.
Define water-level and discharge fields carefully
A field called “level” is insufficient. State whether the value is depth below a measuring point, elevation against a datum or another defined quantity. Record the measuring point and its relationship to the asset. Specify the sign convention and units. A positive number can mean a deeper water surface in one convention and a higher elevation in another.
For discharge, record the quantity, unit, method and boundary. Distinguish instantaneous flow from interval-average flow and totalised volume. Preserve instrument scaling and the configuration used during collection. If a dataset contains both measured and estimated values, provide a field that identifies the distinction without requiring users to infer it from formatting or cell colour.
Example field | Meaning and type | Validation concept |
|---|---|---|
well_id | Stable text identifier for the borehole | Must exist in the well register |
test_id | Stable text identifier for the event | Must connect to one defined test configuration |
observed_at | Timestamp with explicit timezone | Must be interpretable and ordered for its series |
level_depth_m | Depth below the named measuring point | Reference and sign convention required |
flow_m3_h | Discharge at the stated measurement boundary | Method and quality status required |
quality_code | Controlled text code | Must exist in the dictionary’s code list |
source_record_id | Link to original observation evidence | Must resolve to a preserved source |
Document missing values as information
A blank cell can mean not measured, not applicable, unavailable, rejected or unknown. These meanings should not be merged. Define the permitted missing-data codes and how calculations treat them. Never use zero as a generic missing-value substitute when zero is a physically meaningful observation.
Explain whether a field is mandatory for every record or conditional on the record type. For example, a manual water-level observation may not have an automatic logger channel identifier, but it still needs its method and source record. Validation rules should enforce the meaning of the data rather than demand irrelevant values solely to fill every cell.
Preserve events that explain the time series
A numerical series needs an event log. Record pump starts and stops, rate changes, valve changes, instrument work, interruptions and relevant field observations. Give events timestamps and links to the affected asset or test. Where the exact time is uncertain, record that uncertainty instead of inventing precision.
Events should be factual where possible. “Pump stopped at the recorded time” is different from “pump failed because of low groundwater.” The latter is a diagnosis requiring evidence. Keep observation, initial interpretation and final disposition separate so that a later review can understand how the conclusion developed.
Select formats for both use and preservation
Provide a human-readable report alongside structured data where future analysis is expected. CSV can be useful for simple tables, but the dictionary must define encoding, delimiter, decimal convention, date format and missing values. Spreadsheet files may carry formulas and formatting, but export the values and preserve the calculation logic when reproducibility matters.
Do not assume that a file opens correctly merely because it has a familiar extension. Test the handover in the recipient’s likely software. Check that identifiers retain leading zeros, timestamps are not reinterpreted, long numbers are not rounded and units remain visible. These are common import risks that can change meaning without producing an obvious error message.
Build a package index
The index should list each file, its purpose, revision, relationship to other records and whether it is raw, corrected, derived or issued. Identify the current operator instruction and the evidence package supporting it. Include a checksum record when file-integrity checking is part of the project’s preservation process.
Keep the index short enough to use. It should guide the recipient to the right material, not reproduce every table in the project. A useful acceptance question is whether someone unfamiliar with the folder can locate the current operating duty, the raw test observations and the dictionary within a few deliberate steps. If not, improve the structure rather than adding another cover note.
Test the dictionary with a real import
Ask a recipient to import a small representative dataset using only the handover documentation. Include an ordinary observation, a missing value, a flagged value, a manual measurement and a relevant event. Compare the imported result with the source. Record where the recipient needed clarification.
The test is successful when the values and meanings survive the transfer. It should detect ambiguous units, unsupported characters, timezone assumptions and accidental numeric conversion of identifiers. Resolve the dictionary or file-format issue before issuing the full dataset. An import demonstration provides stronger evidence of usability than a statement that the package is “machine-readable.”
Fictional dictionary correction
A dataset contains a column named WL with values around 30. One recipient interprets it as water-level elevation; another treats it as depth below casing. The numbers alone cannot resolve the disagreement. The correct response is to recover the measurement reference from the field record, rename or clearly define the field and document the correction.
Do not infer the convention from whether a plotted line looks plausible. Preserve the original file and issue a corrected dictionary and, if necessary, a revised dataset. Then assess reports and calculations that consumed the ambiguous field.
Specify the observation table before collection
Define the observation table around one clear record type. For a time series, a row might represent one observation from one instrument at one timestamp. If several quantities are stored in the same row, document whether they were measured simultaneously or assembled from different sources. A wide spreadsheet can conceal timing differences that matter to interpretation.
Identify the primary key or uniqueness rule. A combination of test, instrument and timestamp may be suitable in some projects, but duplicate timestamps can occur and must have a defined treatment. Do not delete duplicates automatically without checking whether they represent repeated measurements, imported copies or different observation methods.
Define required fields and validation rules in advance. This reduces the temptation to invent missing metadata after collection. Where field conditions prevent a required observation, record the exception. The dataset should preserve what was actually collected and the reason for any limitation.
Design an event table that supports interpretation
Events need their own identity, time, affected asset, event type and description. Separate a recorded event from its later diagnosis. For example, a pump stop can be recorded immediately, while the cause may be established after investigation. Preserve both records and their relationship rather than overwriting the original description.
Define event start and end conventions. An interruption duration can be calculated incorrectly if one person records the time the fault began and another records the time it was noticed. Where the exact time is uncertain, include a qualification. Do not imply second-level precision when the evidence supports only an approximate interval.
Connect events to affected observations where possible. This makes it easier to interpret a rate change, instrument replacement or reference alteration. The link should remain usable after sorting or exporting the tables, so avoid relying only on row positions or spreadsheet colours.
Document numeric conventions and units
Specify decimal notation, thousands separators and unit fields. A value such as 1,250 can be interpreted differently under different regional conventions. Use a consistent machine-readable format and test it in the receiving environment. The dictionary should remove ambiguity rather than require the recipient to infer the author’s locale.
Avoid placing units only in a decorative report heading. Each dataset field needs a defined unit, and mixed-unit fields should be avoided or explicitly represented. If a unit conversion is applied, preserve the original value or source and document the conversion. This allows future reviewers to verify that a reported rate was not mis-scaled.
State precision and rounding practice. Retain sufficient precision for the intended calculation without implying that every displayed digit represents measurement accuracy. Formatting a value to two decimals does not establish an uncertainty of one hundredth of the unit. Keep measurement uncertainty and display precision distinct.
Define quality-code behaviour in calculations
The dictionary should explain not only what a quality code means but how downstream calculations are expected to treat it. An accepted value, an estimate and a suspect value may all be visible in a chart while only some are used in a particular analysis. Record that analysis rule separately from the raw quality designation.
Avoid hard-coding exclusion through formatting. If a calculation omits a record, it should do so through an explicit rule or documented selection. This supports reproducibility and allows another analyst to test whether the exclusion materially affects the result. A hidden filter is not an adequate correction record.
When quality codes change after review, issue the dataset with a new version or a traceable change record. Preserve the earlier state where it supported an issued result. The recipient should be able to reconstruct which data treatment was used for that result.
Package equipment and configuration metadata
Include instrument make or model, identity, range, relevant settings, verification information and installation location where needed to interpret the observations. The exact fields should reflect the equipment and project. Do not collect a serial number while omitting the scaling factor that determines the recorded value.
Record effective dates for configuration changes. If a transmitter range changes during a campaign, the data-processing rule may also need to change. A single undated configuration sheet cannot adequately describe both periods. Link each observation interval to the applicable configuration.
Keep equipment documentation accessible without making the main dataset unwieldy. The register can reference the relevant file and version. Test that those references resolve in the delivered package, particularly when files are moved from a working directory into an archive.
Test a round trip through the recipient software
Export a representative subset, import it into the recipient’s tool and export it again. Compare identifiers, timestamps, numeric values and missing-data codes. This round-trip exercise can reveal automatic date conversion, lost leading zeros, character-encoding problems or precision changes.
Include difficult cases intentionally: a missing observation, a negative value where meaningful, a long identifier, a timestamp near a date boundary and a qualified record. The purpose is not to manufacture failure but to test the conventions that are most likely to be misinterpreted.
Document any required import settings in the handover instructions. If the recipient must select a delimiter or prevent automatic date interpretation, say so. A file that is correct only under undocumented import settings is not a dependable handover dataset.
Maintain dictionary versions with dataset versions
A changed field definition can alter meaning even when the numeric file is unchanged. Version the dictionary and record which dataset versions it describes. If a field is renamed, document the mapping. If its meaning changes, avoid reusing the old name without an explicit version distinction.
Review downstream outputs when definitions change. A corrected sign convention or unit may affect charts, calculations and conclusions. The dictionary is part of the evidence chain, not merely an appendix. Treat material definition changes through the same impact-review process as data corrections.
Keep a readable change history. The recipient should be able to understand why a new dictionary was issued and whether earlier records require reinterpretation. This supports long-term reuse and prevents different teams from maintaining incompatible private definitions of the same field.
Data-handover acceptance questions
Can the recipient identify every asset and event, interpret every field, distinguish raw and processed values and understand missing-data codes? Can a representative calculation be reproduced using the delivered values and definitions? Do file references resolve after the package is moved to its destination?
If any answer is uncertain, identify the specific defect and correct it before declaring the dataset fully usable. Do not disguise that gap with an invented field value or an undocumented assumption.
Questions about data handover
Is a PDF enough for future analysis
A PDF can communicate conclusions and preserve a readable report. When future numerical analysis is expected, provide structured observations and their definitions as well. The report and dataset serve different purposes.
Can spreadsheet colours identify rejected readings
Colours may help a reader, but they should not be the only machine-readable indication. Use explicit quality fields with documented codes and preserve the reason for rejection or qualification.
Should raw and corrected values share one column
Keep them distinguishable. A corrected view may be convenient, but the recipient must be able to recover the original value and understand the transformation.
What makes the handover complete
Completeness depends on the agreed scope. At minimum, the recipient should be able to identify the assets and records, interpret every delivered field, locate the current conclusions and reproduce the material calculations intended for future use.
Source notes
Content-structure method attribution: Eugen Ullrich, eullrich.com, GEO Content Engineering, CC BY 4.0. No ranking or AI citation guarantee is implied.