Request Flow
How Bingsan processes HTTP requests
Request Flow
Understanding how Bingsan processes requests helps with debugging and performance optimization.
HTTP Request Lifecycle
┌──────────────┐
│ Client │
└──────┬───────┘
│ HTTP Request
▼
┌──────────────────────────────────────────────┐
│ Fiber HTTP Server │
├──────────────────────────────────────────────┤
│ 1. Request ID Middleware │
│ 2. Recovery Middleware (panic handling) │
│ 3. Prometheus Metrics Middleware │
│ 4. CORS Middleware │
│ 5. Logger Middleware │
│ 6. Auth Middleware (if enabled) │
├──────────────────────────────────────────────┤
│ Route Handler │
├──────────────────────────────────────────────┤
│ Database Operations │
├──────────────────────────────────────────────┤
│ Event Publishing │
└──────────────────────────────────────────────┘
│
▼ HTTP Response
┌──────────────┐
│ Client │
└──────────────┘Middleware Stack
1. Request ID
Assigns a unique ID to each request for tracing. Returned in X-Request-ID header.
2. Recovery
Catches panics and returns 500 Internal Server Error.
3. Prometheus Metrics
Records request metrics:
iceberg_catalog_http_requests_total- Countericeberg_catalog_http_request_duration_seconds- Histogram
Paths /health, /ready, /metrics are excluded.
4. CORS
Allows cross-origin requests with configurable origins and methods.
5. Logger
Logs request details including method, path, status, latency, and request ID.
6. Authentication
When enabled, validates bearer tokens. Returns 401 if token is invalid or missing.
Table Commit Flow
The most complex operation is a table commit:
┌──────────────┐
│ Client │
└──────┬───────┘
│ POST /v1/namespaces/{ns}/tables/{table}
▼
┌──────────────────────────────────────────────┐
│ CommitTable Handler │
├──────────────────────────────────────────────┤
│ 1. Parse request body │
│ 2. Validate table identifier │
│ 3. Acquire table lock (PostgreSQL advisory) │
│ 4. Begin transaction │
│ 5. Load current metadata │
│ 6. Check requirements │
│ 7. Apply updates │
│ 8. Write new metadata file (S3/GCS) │
│ 9. Update database │
│ 10. Commit transaction │
│ 11. Release lock │
│ 12. Publish event │
└──────────────────────────────────────────────┘Lock Acquisition
-- Acquire advisory lock on table
SELECT pg_advisory_xact_lock($1);The lock is held for the duration of the transaction.
Requirement Checking
Each requirement is validated against the current table state. If any requirement fails, the entire commit is rejected.
Update Application
Updates are applied in order, supporting operations like add-snapshot, set-current-schema, etc.
Error Handling
Client Errors (4xx)
- 400 Bad Request: Invalid JSON, missing fields
- 401 Unauthorized: Missing or invalid token
- 404 Not Found: Namespace/table doesn't exist
- 409 Conflict: Requirement check failed
Server Errors (5xx)
- 500 Internal Server Error: Unexpected errors, database failures
- 503 Service Unavailable: Database unavailable
Error Response Format
{
"error": {
"message": "Requirement not met: current-schema-id is 1, expected 0",
"type": "CommitFailedException",
"code": 409
}
}Performance Considerations
Connection Pooling
Database connections are pooled via pgx/v5 with configurable max/min connections.
Goroutine-per-Request
Each request runs in its own goroutine with automatic scheduling and efficient memory usage.
JSON Serialization
Uses goccy/go-json for faster JSON encoding/decoding.
Debugging
Request Tracing
Use the request ID to trace through logs:
grep "request_id.*abc-123" /var/log/bingsan.logSlow Request Analysis
Check the request duration in metrics:
histogram_quantile(0.99,
rate(iceberg_catalog_http_request_duration_seconds_bucket{path="/v1/namespaces/{namespace}/tables/{table}"}[5m])
)