Data engineering · Oil and gas
OSDU data platform implementation
We carry out OSDU data platform implementation for operators and national oil companies: gap analysis against your data, mapping to OSDU schemas, ingestion pipelines using manifests, entitlements and legal tagging, integration with applications and migration of legacy subsurface and well data. We work with the OSDU platform your cloud provider or vendor supplies, and you own the integration code and the mappings.
Checklist · Exploration data
Subsurface data management and OSDU readiness checklist
A checklist for exploration and subsurface data management projects: inventory, standards, quality, entitlements, data rooms, migration and the interfaces an OSDU data platform expects.- 01Eight areas to assess before you choose or build a subsurface data platform.
- 02The metadata and identifiers that make wells, surveys and licences findable.
- 03Entitlement and data room items that legal and partners ask about.
- 04Seven questions to put to any vendor or development partner.
Free download
Get it on screen in a few seconds.
What the system does, module by module.
- 01
Readiness and gap analysis
Assessment of your data, applications, skills and governance against OSDU, with a prioritised plan of which domains to bring first. - 02
Platform and environment design
Choice of platform distribution, data partitions, environments, networking and operations model, in line with security and residency needs. - 03
Schema and reference data mapping
Mapping of existing models, such as PPDM databases and interpretation project data, to OSDU schemas and reference data, with documented decisions for gaps. - 04
Manifest-based ingestion
Pipelines that extract, transform and load data using OSDU manifests and workflows, with validation, error handling and reconciliation reports. - 05
Entitlements and legal tagging
A design for data partitions, groups, access control lists and legal tags that reflects licences, partners and jurisdictions. - 06
Wellbore, seismic and reservoir data services
Loading and access to wellbore, log, trajectory, seismic and reservoir model data through the domain data management services. - 07
Application integration
Connection of interpretation, engineering and analytics applications to the platform through its APIs, with identity and permissions carried through. - 08
Search, catalogue and analytics
Search and discovery on platform data, notebooks and pipelines for analytics and machine learning on governed subsurface data. - 09
Migration, operations and governance
Phased migration of legacy repositories, platform monitoring and cost control, and data governance roles and processes.
Reference architecture.
Standards and interfaces
What we design against.
- OSDU Data Platform Standard
- The Open Group's standard defining the platform's services, schemas and behaviour, with the community implementations by cloud providers and vendors.
- OSDU schemas and reference data
- The well-known schemas and the governed reference value lists that data is mapped to.
- Energistics standards
- WITSML, PRODML and RESQML for exchanging data with sources and applications.
- PPDM data model
- A common source model for legacy databases, which mappings start from.
- Identity and access standards
- OpenID Connect and the cloud provider's identity services for authentication and groups.
- Data residency and legal requirements
- Legal tagging of data by origin, jurisdiction and contract.
Data model
The core entities.
- Record and schema
- OSDU kind, version, content, relationships and the source it was loaded from.
- Mapping specification
- Source field, target attribute, transformation, reference data used and decisions taken.
- Manifest and ingestion run
- Content, validation results, errors, record counts and reconciliation.
- Entitlement group and ACL
- Groups, members, access levels and the records they control.
- Legal tag
- Origin, jurisdictions, contract, expiry and export rules.
- Dataset and file
- Location, format, checksum and the record that describes it.
- Application registration
- Application, identity, scopes and integration status.
When custom development is the right call.
Controls designed in, not added later.
Least-privilege entitlements
Groups and access control lists follow roles and licences, with legal tags enforcing jurisdiction and contract limits.Identity integration
Single sign-on and group sync with the company directory, with service identities for pipelines and applications.Data integrity and lineage
Checksums, versioned records and a log of ingestion runs, so any record can be traced to its source and load.Network and residency controls
Private connectivity and regional deployment aligned with law and policy.Operational monitoring
Monitoring of platform health, ingestion errors and unusual access.
From discovery to a system you run.
- 01
Readiness and priorities
We assess data, applications and governance, agree the first domains and use cases and define success measures. - 02
Platform and entitlements design
The platform environment, partitions, groups and legal tag model are designed with IT, security, legal and data management. - 03
Map and ingest the first domain
One domain, often wells and logs, is mapped and loaded end to end, and reconciled to the source. - 04
Integrate applications and analytics
Key applications and analytics workflows connect to the platform and use the loaded data. - 05
Migrate further domains and run
More domains and legacy sources follow, with governance, operations and cost management in place.
Relevant work
Where to see our work.
Common questions.
- Do you provide the OSDU platform itself?
- No. The platform comes from your cloud provider or a vendor. We work on what makes it useful: mapping and ingesting your data, designing entitlements, integrating applications and migrating legacy sources.
- Where should we start?
- With one domain and one use case that matter to the business, such as well and log data for a specific asset. A small, complete slice shows the effort, the data quality issues and the value before you commit more.
- How do you handle data that does not fit an OSDU schema?
- We document the gap and choose between using extension attributes, proposing a schema change through the community or keeping the data in a linked store. The decision is recorded in the mapping specification.
- What about licensed and partner data?
- Legal tagging and entitlements are designed from the contracts, so access follows licence terms and jurisdiction, and changes in licence status are reflected in access.
- Who owns the mappings and code?
- You do. The mappings, the pipelines and the integration code stay yours.