9e86c93fdaaa3bd9bff4cf878342a7be4a1a0992
11 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
vsecoder
|
9e86c93fda |
test(feed): end-to-end integration + two-node propagation (Phase B hardening)
Adds two integration-test files that exercise the full feed stack over
real HTTP requests, plus a fix to the publish signature model that the
EXIF scrubbing test surfaced.
Bug fix — api_feed.go publish signature flow
Previously: server scrubbed the attachment → computed content_hash
over the SCRUBBED bytes → verified the author's signature against
that hash. But the client, not owning the scrubber, signs over the
RAW upload. The two hashes differ whenever scrub touches the bytes
(which it always does for images), so every signed upload with an
image was rejected as "signature invalid".
Fixed order:
1. decode attachment from base64
2. compute raw_content_hash over Content + raw attachment
3. verify author's signature against raw_content_hash
4. scrub attachment (strips EXIF / re-encodes)
5. compute final_content_hash over Content + scrubbed attachment
6. return final hash in response for the on-chain CREATE_POST tx
The signature proves the upload is authentic; the final hash binds
the on-chain record to what readers actually download.
node/feed_e2e_test.go
In-process harness: real BadgerDB chain + feed mailbox + media
scrubber + httptest.Server with RegisterFeedRoutes. Tests drive
it via real http.Post / http.Get so rate limiters, auth, scrubber,
and handler code all run on the happy path.
Tests:
- TestE2EFullFlow — publish → CREATE_POST tx → body fetch → view
bump → stats → author list → soft-delete → 410 Gone on re-fetch
- TestE2ELikeUnlikeAffectsStats — on-chain LIKE_POST bumps /stats,
liked_by_me reflects the caller
- TestE2ETimeline — follow graph, merged timeline newest-first
- TestE2ETrendingRanking — likes × 3 + views puts hot post at [0]
- TestE2EForYouFilters — excludes own posts + followed authors +
already-liked posts; surfaces strangers
- TestE2EHashtagSearch — tag returns only tagged posts
- TestE2EScrubberStripsEXIF — injects SUPERSECRETGPS canary into a
JPEG APP1 segment, uploads via /feed/publish, reads back — asserts
canary is GONE from stored attachment. This is the privacy-critical
regression gate: if it ever breaks, GPS coordinates leak.
- TestE2ERejectsMIMEMismatch — PNG labelled as JPEG → 400
- TestE2ERejectsBadSignature — wrong signer → 403
- TestE2ERejectsStaleTimestamp — 1-hour-old ts → 400 (anti-replay)
node/feed_twonode_test.go
Simulates two independent nodes sharing block history (gossip via
same-block AddBlock on both chains). Verifies the v2.0.0 design
contract: chain state replicates, but post BODIES live only on the
hosting relay.
Tests:
- TestTwoNodePostPropagation — Alice publishes on A; B's chain sees
the record; B's HTTP /feed/post/{id} returns 404 (body is A's);
fetch from A succeeds using hosting_relay field from B's chain
lookup. Documents the client-side routing contract.
- TestTwoNodeLikeCounterSharedAcrossNodes — Bob likes from Node B;
both A's and B's /stats show likes=1. Proves engagement aggregates
are chain-authoritative, not per-relay.
- TestTwoNodeFollowGraphReplicates — FOLLOW tx propagates, /timeline
on B returns A-hosted posts with metadata (no body, as designed).
Coverage summary
Publish flow (sign → scrub → hash → store): ✓
CREATE_POST on-chain fee accounting: ✓
Like / Unlike counter consistency: ✓
Follow graph → timeline merge: ✓
Trending ranking by likes × 3 + views: ✓
For You exclusion rules (self, followed, liked): ✓
Hashtag inverted index: ✓
View counter increment + stats aggregate: ✓
Soft-delete → 410 Gone: ✓
Metadata scrubbing (EXIF canary): ✓
MIME mismatch rejection: ✓
Signature authenticity: ✓
Timestamp anti-replay (±5 min window): ✓
Two-node block propagation: ✓
Cross-node body fetch via hosting_relay: ✓
Likes aggregation across nodes: ✓
All 7 test packages green: blockchain consensus identity media node
relay vm.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
vsecoder
|
f885264d23 |
feat(media): mandatory metadata scrubbing on /feed/publish + FFmpeg sidecar
Every photo from a phone camera ships with an EXIF block that leaks:
GPS coordinates, camera model + serial, original timestamp, software
name, author/copyright fields, sometimes an embedded thumbnail that
survives cropping. For a social feed positioned as privacy-friendly
we can't trust the client alone to scrub — a compromised build,
a future plugin, or a hostile fork would simply skip the step and
leak authorship data.
So: server-side scrub is mandatory for every /feed/publish upload.
New package: media
media/scrub.go
- Scrubber type with Scrub(ctx, bytes, claimedMIME) → (clean, actualMIME)
- ScrubImage handles JPEG/PNG/GIF/WebP in-process: decodes, optionally
downscales to 1080px max-dim, re-encodes as JPEG Q=75. Stdlib
jpeg.Encode emits ZERO metadata → scrub is complete by construction.
- Sidecar client (HTTP): posts video/audio bytes to an external
FFmpeg worker at DCHAIN_MEDIA_SIDECAR_URL
- Magic-byte MIME detection: rejects uploads where declared MIME
doesn't match actual bytes (prevents a PDF dressed as image/jpeg
from bypassing the scrubber)
- ErrSidecarUnavailable: explicit error when video arrives but no
sidecar is wired; operator opts in to fallback via
--allow-unscrubbed-video (default: reject)
media/scrub_test.go
- Crafted EXIF segment with "SECRETGPS-…Canon-EOS-R5" canary —
verifies the string is gone after ScrubImage
- Downscale test (2000×1000 → 1080×540, aspect preserved)
- MIME-mismatch rejection
- Magic-byte detector sanity table
FFmpeg sidecar — new docker/media-sidecar/
Tiny Go HTTP service (~180 LOC, no non-stdlib deps) that shells out
to ffmpeg with -map_metadata -1 + -map 0:v -map 0:a? to guarantee
only video + audio streams survive (no subtitles, attached pictures,
or data channels that could carry hidden info).
Re-encode profile:
video → H.264 CRF 28 preset=fast, Opus 64k, MP4 faststart
audio → Opus 64k, Ogg container
Dockerfile: two-stage build (Go → alpine+ffmpeg), ~90 MB image, non-
root user, /healthz endpoint for compose probes.
Node reaches it via DCHAIN_MEDIA_SIDECAR_URL. Without it, video uploads
are rejected with 503 unless operator sets DCHAIN_ALLOW_UNSCRUBBED_VIDEO.
/feed/publish wiring
- cfg.Scrubber is a required dependency
- Before storing post body we call scrubber.Scrub(); attachment bytes
+ MIME are replaced with the cleaned version
- content_hash is computed over the SCRUBBED bytes — so the on-chain
CREATE_POST tx references exactly what readers will fetch
- EstimatedFeeUT uses the scrubbed size, so author's fee reflects
actual on-disk cost
- Content-type mismatches → 400; sidecar unavailable for video → 503
Flags / env vars
--feed-db / DCHAIN_FEED_DB (existing)
--feed-ttl-days / DCHAIN_FEED_TTL_DAYS (existing)
--media-sidecar-url / DCHAIN_MEDIA_SIDECAR_URL (NEW)
--allow-unscrubbed-video / DCHAIN_ALLOW_UNSCRUBBED_VIDEO (NEW; default false)
Client responsibilities (for reference — client work lands in Phase C)
Even with server-side scrub, the client should still compress aggressively
BEFORE upload, because:
- upload time is ~N× larger for unscrubbed media (mobile networks)
- the server's 256 KiB MaxPostSize is a HARD cap — oversized uploads
are rejected, not silently truncated
- the on-chain fee is size-based, so users pay for every byte the
client didn't bother to shrink
Recommended client pipeline:
images → expo-image-manipulator: resize max-dim 1080px, WebP or
JPEG quality 50-60
videos → react-native-compressor: H.264 CRF 28, 720p max, 64k audio
audio → expo-audio's default Opus 32k (already compressed)
Documented in docs/media-sidecar.md (added later with Phase C PR).
Tests
- go test ./... green across 6 packages (blockchain consensus identity
media relay vm)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
vsecoder
|
126658f294 |
feat(feed): relay body storage + HTTP endpoints (Phase B of v2.0.0)
Phase A (the previous commit) added the on-chain foundations. Phase B
is the off-chain layer: post bodies live in a BadgerDB-backed feed
mailbox, and a full HTTP surface makes the feed usable from clients.
New components
relay/feed_mailbox.go (+ tests)
- FeedPost: body + content-type + attachment + hashtags + thread refs
- Store / Get / Delete with TTL-bounded eviction (30 days default)
- View counter (IncrementView / ViewCount) — off-chain because one
tx per view would be nonsense
- Hashtag inverted index: auto-extracts #tokens from content on
Store, lowercased + deduped + capped at 8/post
- Author chrono index: PostsByAuthor returns newest-first IDs
- RecentPostIDs: scan-by-age helper used by trending/foryou
node/api_feed.go
POST /feed/publish — author-signed body upload, returns
post_id + content_hash + size +
hashtags + estimated fee for the
follow-up on-chain CREATE_POST tx
GET /feed/post/{id} — fetch body (respects on-chain soft
delete, returns 410 when deleted)
GET /feed/post/{id}/stats — {views, likes, liked_by_me?}
POST /feed/post/{id}/view — bump the counter
GET /feed/author/{pub} — chain-authoritative post list
enriched with body + stats
GET /feed/timeline — merged feed from people the user
follows (reads chain.Following,
fetches each author's recent posts)
GET /feed/trending — top-scored posts in last 24h
(score = likes × 3 + views)
GET /feed/foryou — simple recommendations: recent posts
minus authors the user already
follows, already-liked posts, and
own posts; ranked by engagement
GET /feed/hashtag/{tag} — posts tagged with the given #tag
cmd/node/main.go wiring
- --feed-db flag (DCHAIN_FEED_DB) + --feed-ttl-days (DCHAIN_FEED_TTL_DAYS)
- Opens FeedMailbox + registers FeedRoutes alongside RelayRoutes
- Threads chain.Post / LikeCount / HasLiked / PostsByAuthor / Following
into FeedConfig so HTTP handlers can merge on-chain metadata with
off-chain body+stats.
Auth & safety
- POST /feed/publish: Ed25519 signature over "publish:<post_id>:
<content_sha256_hex>:<ts>"; ±5-minute skew window for anti-replay.
- content_hash binds body to the on-chain tx — you can't publish
body-A off-chain and commit hash-of-body-B on-chain.
- Writes wrapped in withSubmitTxGuards (rate-limit + size cap), reads
in withReadLimit — same guards as /relay.
Trending / recommendations
- V1 heuristic (likes × 3 + views) + time window. Documented as
v2.2.0 "Feed algorithm" candidate for a proper ranking layer
(half-life decay, follow-of-follow boost, hashtag collaborative).
Tests
- Store round-trip, size enforcement, hashtag indexing (case-insensitive
+ dedup), view counter increments, author chrono order, delete
cleans all indices, RecentPostIDs time-window filter.
- Full go test ./... is green (blockchain + consensus + identity +
relay + vm all pass).
Next (Phase C): client Feed tab — composer, timeline, post detail,
profile follow, For You + Trending screens.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
vsecoder
|
88848efa63 |
feat(chain): remove channels, add social feed (Phase A of v2.0.0)
Replaces the channel/membership model with a VK/Twitter-style feed:
public posts, follow graph, likes. Views are deliberately off-chain
(counted by the hosting relay, Phase B).
Removed
- EventCreateChannel, EventAddMember
- CreateChannelPayload, AddMemberPayload, ChannelMember
- prefixChannel, prefixChanMember
- chain.Channel(), chain.ChannelMembers()
- node/api_channels.go
- GetChannel, GetChannelMembers on ExplorerQuery
Added
- Events: CREATE_POST, DELETE_POST, FOLLOW, UNFOLLOW, LIKE_POST, UNLIKE_POST
- Payloads: CreatePostPayload, DeletePostPayload, FollowPayload,
UnfollowPayload, LikePostPayload, UnlikePostPayload
- Stored shape: PostRecord (author, size, hash, hosting relay, timestamp,
reply/quote refs, soft-delete flag, fee paid)
- State prefixes: post:, postbyauthor:, follow:, followin:, like:, likecount:
- Queries: Post(), PostsByAuthor(), Following(), Followers(),
LikeCount(), HasLiked()
- Cached like counter via bumpLikeCount helper
Pricing
- BasePostFee = 1000 µT (aligned with MinFee block-validation floor)
- PostByteFee = 1 µT/byte of compressed content
- Total fee credited in full to HostingRelay pub (storage compensation)
- MaxPostSize = 256 KiB
Integrity
- CREATE_POST validates content_hash length (32 B) and size range
- DELETE_POST restricted to post.Author
- Duplicate FOLLOW / LIKE rejected
- reply_to and quote_of mutually exclusive
Tests
- TestFeedCreatePost: post stored, indexed, host credited
- TestFeedInsufficientFee: underpaid post is skipped
- TestFeedFollowUnfollow: follow graph round-trips via forward + inbound indices
- TestFeedLikeUnlike: like toggles with dedup, counter stays accurate
- TestFeedDeletePostByOther: non-author deletion rejected
This is Phase A (chain-layer). Phase B adds the relay feed-mailbox
(post bodies + gossipsub) and HTTP endpoints. Phase C adds the client
Feed tab.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
vsecoder
|
f2cb5586ca |
fix(relay): require signed Ed25519 auth on DELETE /relay/inbox/{id}
Previously the endpoint accepted an unauthenticated DELETE with just
?pub=X — anyone who knew (or enumerated) a pub could wipe that pub's
entire inbox, a trivial griefing vector. Now the handler requires a
JSON body with {ed25519_pub, sig, ts} where sig signs
"inbox-delete:<envID>:<pub>:<ts>" under the Ed25519 privkey. The
server then looks up the identity on-chain and verifies that the
registered X25519 public key matches the ?pub= query — closing the
gap between "I can sign" and "my identity owns this mailbox."
Timestamp window: ±300s to prevent replay of captured DELETEs.
Wires RelayConfig.ResolveX25519 via chain.Identity() in cmd/node/main.go.
When ResolveX25519 is nil the endpoint returns 503 (feature unavailable)
rather than silently allowing anonymous deletes.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
v1.0.2
|
||
|
vsecoder
|
15d0ed306b |
fix(ws): hard-deny inbox:* / typing:* when authX is empty
The WS topic-auth check had a soft-fail fallback: if the authenticated identity had no registered X25519 public key (authX == ""), the topic-ownership check was skipped and the client could subscribe to any inbox:* or typing:* topic. Exploit: register an Ed25519 identity without an X25519 key, subscribe to the victim's inbox topic, receive their envelope notifications. Now both topics hard-require a registered X25519. Clients must call REGISTER_KEY (publishing X25519) before subscribing. The scope is narrow — only identities that haven't completed REGISTER_KEY yet could have exploited this — but a hard fail is still correct. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
vsecoder
|
8082dd0bf7 |
fix(node): rate-limit relay HTTP endpoints
Relay routes were not wrapped in any guards — /relay/broadcast accepted unlimited writes from any IP, and /relay/inbox could be scraped at line rate. Combined with the per-recipient FIFO eviction (MailboxPerRecipientCap=500), an unauthenticated attacker could wipe a victim's real messages by spamming 500 garbage envelopes. This commit wraps writes in withSubmitTxGuards (10/s per IP + 256 KiB body cap) and reads in withReadLimit (20/s per IP) — the same limits already used for /api/tx and /api/address. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
vsecoder
|
32eec62ba4 |
fix(chain): RELAY_PROOF dedup by envelopeID + sticky BlockContact
RELAY_PROOF previously had no per-envelope dedup — every relay that saw the gossipsub re-broadcast could extract the sender's FeeSig from the envelope and submit its own RELAY_PROOF claim with its own RelayPubKey. The tx-ID uniqueness check didn't help because tx.ID = sha256(relayPubKey||envelopeID)[:16], which is unique per (relay, envelope) pair. A malicious mesh of N relays could drain N× the fee from the sender's balance for a single message. Fix: record prefixRelayProof:<envelopeID> on first successful apply and reject subsequent claims for the same envelope. CONTACT_REQUEST previously overwrote any prior record (including a blocked one) back to pending, letting spammers unblock themselves by paying another MinContactFee. Now the handler reads the existing record first and rejects the tx with "recipient has blocked sender" when prev.Status == ContactBlocked. Block becomes sticky. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
vsecoder
|
78d97281f0 |
fix(relay): canonicalise envelope ID and timestamp on mailbox.Store
The mailbox previously trusted the client-supplied envelope ID and SentAt,
which enabled two attacks:
- replay via re-broadcast: a malicious relay could resubmit the same
ciphertext under multiple IDs, causing the recipient to receive the
same plaintext repeatedly;
- timestamp spoofing: senders could back-date or future-date messages
to bypass the 7-day TTL or fake chronology.
Store() now recomputes env.ID as hex(sha256(nonce||ct)[:16]) and
overwrites env.SentAt with time.Now().Unix(). Both values are mutated
on the envelope pointer so downstream gossipsub publishes agree on the
normalised form.
Also documents /relay/send as non-E2E — the endpoint seals with the
relay's own key, which breaks end-to-end authenticity. Clients wanting
real E2E should POST /relay/broadcast with a pre-sealed envelope.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
v1.0.1
|
||
|
vsecoder
|
546d2c503f |
chore(release): clean up repo for v0.0.1 release
Excluded from release bundle:
- CONTEXT.md, CHANGELOG.md (agent/project working notes)
- client-app/ (React Native messenger — tracked separately)
- contracts/hello_go/ (unused standalone example)
Kept contracts/counter/ and contracts/name_registry/ as vm-test fixtures
(referenced by vm/vm_test.go; NOT production contracts).
Docs refactor:
- docs/README.md — new top-level index with cross-references
- docs/quickstart.md — rewrite around single-node as primary path
- docs/node/README.md — full rewrite, all CLI flags, schema table
- docs/api/README.md — add /api/well-known-version, /api/update-check
- docs/contracts/README.md — split native (Go) vs WASM (user-deployable)
- docs/update-system.md — new, full 5-layer update system design
- README.md — link into docs/, drop CHANGELOG/client-app references
Build-time version system (inherited from earlier commits this branch):
- node --version / client --version with ldflags-injected metadata
- /api/well-known-version with {build, protocol_version, features[]}
- Peer-version gossip on dchain/version/v1
- /api/update-check against Gitea release API
- deploy/single/update.sh with semver guard + 15-min systemd jitter
v0.0.1
|
||
|
vsecoder
|
7e7393e4f8 |
chore: initial commit for v0.0.1
DChain single-node blockchain + React Native messenger client. Core: - PBFT consensus with multi-sig validator admission + equivocation slashing - BadgerDB + schema migration scaffold (CurrentSchemaVersion=0) - libp2p gossipsub (tx/v1, blocks/v1, relay/v1, version/v1) - Native Go contracts (username_registry) alongside WASM (wazero) - WebSocket gateway with topic-based fanout + Ed25519-nonce auth - Relay mailbox with NaCl envelope encryption (X25519 + Ed25519) - Prometheus /metrics, per-IP rate limit, body-size cap Deployment: - Single-node compose (deploy/single/) with Caddy TLS + optional Prometheus - 3-node dev compose (docker-compose.yml) with mocked internet topology - 3-validator prod compose (deploy/prod/) for federation - Auto-update from Gitea via /api/update-check + systemd timer - Build-time version injection (ldflags → node --version) - UI / Swagger toggle flags (DCHAIN_DISABLE_UI, DCHAIN_DISABLE_SWAGGER) Client (client-app/): - Expo / React Native / NativeWind - E2E NaCl encryption, typing indicator, contact requests - Auto-discovery of canonical contracts, chain_id aware, WS reconnect on node switch Documentation: - README.md, CHANGELOG.md, CONTEXT.md - deploy/single/README.md with 6 operator scenarios - deploy/UPDATE_STRATEGY.md with 4-layer forward-compat design - docs/contracts/*.md per contract |