Project: Event Analytics

Introduction

Product teams track user events—page views, signups, purchases—for funnels and DAU. MongoDB handles high-volume append-only events well. This project ingests events, runs aggregation for DAU and funnels, uses TTL for raw retention, and $merge into summary collections.

Prerequisites

Project Goals

MetricTechnique
Daily active users (DAU)$group by user + day
Event funnel$match sequence per user
Top pages$group by page, $sort, $limit
RetentionTTL on raw events; keep summaries

Events Collection

Indexes:

javascript
db.events.createIndex({ timestamp: 1 })
db.events.createIndex({ userId: 1, timestamp: -1 })
db.events.createIndex({ eventType: 1, timestamp: -1 })
db.events.createIndex(
  { timestamp: 1 },
  { expireAfterSeconds: 7776000 }
)

TTL 7776000 ≈ 90 days—raw events auto-delete; summaries live elsewhere.

DAU for a Day

javascript
db.events.aggregate([
  {
    $match: {
      timestamp: {
        $gte: ISODate("2026-05-28T00:00:00Z"),
        $lt: ISODate("2026-05-29T00:00:00Z")
      }
    }
  },
  { $group: { _id: "$userId" } },
  { $count: "dau" }
])

Top Pages (Last 7 Days)

javascript
db.events.aggregate([
  {
    $match: {
      eventType: "page_view",
      timestamp: { $gte: new Date(Date.now() - 7 * 86400000) }
    }
  },
  { $group: { _id: "$page", views: { $sum: 1 } } },
  { $sort: { views: -1 } },
  { $limit: 10 }
])

Funnel: View → Signup

Simplified two-step funnel per user in a window:

Production funnels often use ordered window logic or a dedicated analytics tool—this teaches pipeline building blocks.

Bucket Pattern (High Volume)

For IoT-scale timestamps, bucket by hour/day — Common Modeling Patterns:

javascript
db.events_hourly.insertOne({
  bucket: ISODate("2026-05-28T10:00:00Z"),
  counts: {
    page_view: 1523,
    signup: 42
  }
})

$merge Daily Summary

Scheduled job (cron, Celery, or Atlas trigger) runs nightly.

Ingest API Sketch (PyMongo)

python
from datetime import datetime, timezone
 
def track_event(events, user_id: str, event_type: str, **metadata):
    doc = {
        "userId": user_id,
        "eventType": event_type,
        "timestamp": datetime.now(timezone.utc),
        "metadata": metadata,
    }
    if event_type == "page_view":
        doc["page"] = metadata.get("page", "/")
    return events.insert_one(doc)

Batch inserts for throughput:

python
events.insert_many(batch, ordered=False)

See bulkWrite — Performance.

Post-Project Checklist

  • TTL index on timestamp verified with test doc
  • DAU and Top pages aggregations documented
  • $merge summary collection populated
  • Ingest path handles duplicate eventId (optional unique index)

FAQ

MongoDB vs ClickHouse/BigQuery?

MongoDB fits operational analytics and moderate volume; warehouse tools win at petabyte SQL reporting.

Real-time dashboards?

Change Streams push to websocket service — Transactions chapter (Change Streams section).

PII in events?

Hash userId, restrict fields, follow retention policy — Security.

Redis for real-time counters?

HyperLogLog or INCR for live counts; MongoDB for durable history — Redis caching patterns.

Deploy full stack?

MongoDB + Redis + API project.