jobqueue

jobqueue

Details

  • Go
  • PostgreSQL
  • pgx
  • Prometheus
  • Docker
  • GitHub Actions

A job queue on plain PostgreSQL that stays correct when workers crash.

An asynchronous job processing service in Go that uses PostgreSQL as both job store and queue: a bounded worker pool claims jobs, retries failures with backoff and recovers work left behind by a crashed process.

Built while learning Go: designed the concurrency model, the job state machine and the delivery guarantees.

Key Achievements

  • Claiming with FOR UPDATE SKIP LOCKED, so any number of workers and processes share one database without handing out the same job
  • Leases with a monotonic fencing token: a worker whose lease expired can't overwrite the result of a newer attempt
  • Exponential backoff with jitter, retryable vs permanent errors, dead-lettering, and a graceful shutdown that releases interrupted jobs without consuming attempts
  • Integration tests under Go's race detector against real PostgreSQL, including three pools running 200 jobs exactly once each
My Projects