Bluefield
Auditable web scraping for compliance
Bluefield gives compliance-conscious teams an audit trail for web data: every scrape returns a signed attestation you can archive as evidence of exactly what was fetched and when. It honors robots.txt and stays on the public surface by default. Bluefield is GDPR-aware — it is not itself a compliance certification.
What is auditable web scraping?
Auditable web scraping produces a verifiable record for each fetch — a signed attestation of the source, content, and time — that you can retain as evidence inside your own audit or compliance process, instead of an unverifiable log entry.
- Audit trail
- A retained set of signed attestations proving what your pipeline fetched and when.
- Attestation archive
- Stored manifests you can produce as evidence later (extended retention on Enterprise).
- Public-surface only
- Bluefield fetches public pages, honors robots.txt, and does not authenticate to or bypass paywalls on third-party sites.
Bluefield capabilities
| Capability | Detail |
|---|---|
| Signed evidence | A verifiable manifest for every fetch, archivable as audit evidence |
| Robots-respecting | robots.txt enforced before any fetch; public-surface only |
| DPA on request | A data processing agreement is available |
| Self-host | Keep scraping and signing entirely in your own cloud |
| Retention | Extended attestation retention on Enterprise plans |
Frequently asked questions
- Is Bluefield GDPR-compliant?
- No — Bluefield is GDPR-aware, not certified. It respects robots.txt, fetches only the public surface, offers a DPA on request, and is US-hosted today. It does not hold SOC 2 or ISO 27001. What it provides is an audit trail you can use inside your own compliance process.
- What can I prove with an attestation?
- That a specific URL returned specific content at a specific time, and that the content has not been altered since. You verify it offline with the published key.
- Can I keep the data in my own infrastructure?
- Yes. Bluefield is self-hostable in Docker with the same API and the same signing pipeline, so data and attestations never leave your cloud.