January 31, 2025 5 min read Data Architecture Series: IMAGE Home

Image Data Architecture: Store, Extract, Search, and Analyze

Image data looks like a file, but production systems need much more: object storage, metadata, thumbnails, quality checks, rights, moderation, and search.

When I work with image data, I keep the original image separate from thumbnails, labels, embeddings, and moderation output. I recommend making each derived artifact traceable back to the source image and model version.

Visual evidence with context

Visual evidence with context Image data shown as a four part decision frame. Visual evidence with context The shape matters because different teams need different proof from the same data. Business moment Customer uploads a damage photo orreceipt image Stored shape Pixels plus labels, source, owner,consent, and review state Decision meaning A visual proof point that peopleor models can inspect Typical handles image_uri label claim_id review_status

Image data becomes operational only when the picture is connected to ownership, labels, and review state.

How photos become reviewable evidence

How photos become reviewable evidence Image data moving from source action into operational and analytical uses. How photos become reviewable evidence Source moment Customer uploads aclaim photo Captured data Image, claim id,device, labels, andreviewer notes Use 1Show visualproof Use 2Prioritizemanual review Use 3Search bylabel Use 4Train qualitychecks Image data becomes operational only when the picture is connected to ownership, labels, andreview state.

Image data becomes operational only when the picture is connected to ownership, labels, and review state.

Storage patterns I would expect

When I talk about "Storage patterns I would expect", I am checking whether Image Data Architecture can be traced back, trusted, and used by someone making a decision.

S3 or Blob or Cloud Storage, PostgreSQL, DynamoDB, MongoDB, OpenSearch, Elasticsearch, vector databases, data lake tables for analytics.

Useful cloud services

ProviderTypical services
AWSS3, Lambda, Rekognition, SageMaker, OpenSearch, Athena.
AzureBlob Storage, Azure AI Vision, Functions, AI Search, Synapse.
Google CloudCloud Storage, Cloud Vision, Dataflow, Vertex AI, BigQuery.

Public example worth knowing

I use "Public example worth knowing" to keep Image Data Architecture grounded in a real system, because abstract patterns are too easy to agree with and too hard to operate.

Facebook Haystack is the classic public example: Facebook described a storage system designed for billions of photos where object storage, metadata, and retrieval efficiency mattered more than ordinary file system thinking.

A business example

I use "A business example" to keep Image Data Architecture grounded in a real system, because abstract patterns are too easy to agree with and too hard to operate.

An insurance claim app uploads photos. The pipeline keeps the original image, creates a review thumbnail, extracts EXIF time and location when allowed, runs damage detection, and indexes claim id, vehicle id, and detected damage type.

Operational code example

When I show "Operational code example", I want the code in Image Data Architecture to reveal the production decision, not just the syntax.

ClaimRoute routeImage(ImageAsset image) {
    if (!image.consentRecorded()) return ClaimRoute.PRIVACY_HOLD;
    if (image.moderationScore() > 0.80) return ClaimRoute.MANUAL_REVIEW;
    if (image.labelConfidence("damage") < 0.85) return ClaimRoute.NEED_BETTER_PHOTO;

    return ClaimRoute.AUTO_ESTIMATE;
}

The routing logic blocks privacy risk first, sends unsafe or uncertain images to review, and only allows automation when the damage label is confident enough.

What I would check first

Continue with

Written by Arunkumar Ganesan.

What I learnt is that image data is not one blob; it is a source asset plus many searchable and reviewable interpretations.

#DataArchitecture #DataEngineering #ETL #CloudArchitecture #IMAGE