Base64 secret padding mismatch

--- name: base64-secret-padding-mismatch description: | Fix for webhook authorization failures caused by Base64 padding character (=) being dropped when copying secrets between systems. Use when: (1) Webhook returns "Unauthorized" or "Server Error" but the secret looks correct, (2) INTERNALAPISECRET, WEBHOOK_SECRET, or similar auth tokens fail validation between Railway/Vercel and Convex/database, (3) Secret comparison fails despite values appearing identical, (4) CLI tools display truncated environment variables. Common with Railway CLI, Vercel CLI, and other deployment platforms that may not display trailing = characters. author: Claude Code version: 1.0.0 date: 2026-01-23 --- # Base64 Secret Padding Mismatch Between Environments ## Problem Webhooks or API calls fail authorization even though the secret "looks correct" when comparing environment variables between systems. The root cause is Base64 padding characters (=) being silently dropped when copying or displaying secrets. ## Context / Trigger Conditions - Webhook handler returns "Unauthorized" or generic "Server Error" - Error occurs in mutation/database call after webhook validation passes - Secret was copy-pasted between systems (e.g., Railway → Convex, Vercel → Supabase) - CLI tools show the secret without trailing = characters - The secret is Base64-encoded (often ends with = or ==) Example error pattern: `` Webhook error: Error: [Request ID: xxx] Server Error at mutationInner (...) ` ## Solution ### Step 1: Identify Base64 Secrets Base64-encoded secrets often: - Are 32-44 characters long - Contain alphanumeric characters plus +, /, and = - End with = or == for padding ### Step 2: Compare Full Values Don't trust CLI display output. Get the raw values: **For Convex:** `bash npx convex env list --prod | grep SECRET_NAME # Output: SECRET_NAME=VbjsOyAxdAuOtBkiSXZiUX2sA4ELpO6FFx3Y8lIDiD4= ` **For Railway:** `bash railway variables | grep SECRET_NAME # Output may truncate: VbjsOyAxdAuOtBkiSXZiUX2sA4ELpO6FFx3Y8lIDiD4 # Note the missing trailing = ` ### Step 3: Fix the Mismatch When setting secrets, explicitly include the full value with padding: `bash # Use single quotes to preserve special characters railway variables set 'INTERNAL_API_SECRET=VbjsOyAxdAuOtBkiSXZiUX2sA4ELpO6FFx3Y8lIDiD4=' # Or for Vercel vercel env add INTERNAL_API_SECRET # Then paste the full value including = ` ### Step 4: Redeploy After fixing the environment variable, trigger a redeploy: `bash railway deployment redeploy --yes # or vercel --prod ` ## Verification 1. Check logs for successful webhook processing 2. Verify the operation that was failing now succeeds 3. For subscription webhooks: check that user status updates correctly ## Example **Scenario:** Polar webhook for subscription activation fails in production. **Symptoms:** - Payment succeeds in Polar - Redirect to app shows ?success=true - User subscription status remains "FREE" - Railway logs show: Webhook error: Error: Server Error **Investigation:** `bash # Convex shows: npx convex env list --prod | grep INTERNAL_API_SECRET # INTERNAL_API_SECRET=VbjsOyAxdAuOtBkiSXZiUX2sA4ELpO6FFx3Y8lIDiD4= # Railway shows (truncated): railway variables | grep INTERNAL # INTERNAL_API_SECRET │ VbjsOyAxdAuOtBkiSXZiUX2sA4ELpO6FFx3Y8lIDiD4 ` **Fix:** `bash railway variables set 'INTERNAL_API_SECRET=VbjsOyAxdAuOtBkiSXZiUX2sA4ELpO6FFx3Y8lIDiD4=' railway deployment redeploy --yes ` ## Notes - This issue is particularly common with secrets generated by openssl rand -base64 32 - Some CLIs and web UIs strip or hide trailing = for display purposes - When in doubt, regenerate the secret and set it in both places simultaneously - Consider using hex encoding (openssl rand -hex 32) to avoid padding issues entirely - The =` padding in Base64 ensures the encoded length is a multiple of 4 ## Prevention When generating and distributing secrets: 1. Generate in one place and immediately copy the full value 2. Use a password manager or secure note to store the canonical value 3. After setting in each system, verify by echoing back or checking logs 4. Consider using non-Base64 formats (hex, UUID) for simpler handling ## Related Scenarios - JWT secrets between auth providers and backends - Stripe/Polar webhook secrets - Internal API authentication between microservices - Any secret that's validated via string comparison