Why
MenkyoKit is our driving-licence study site for foreign residents of Japan: Next.js 14, Supabase, Stripe, with most of the code written by an AI agent from the command line. One day we watched a video about an AI-built site that was emptied a few days after launch: 1.5 million account-takeover tokens, 35,000 e-mail addresses, 4,000 private messages. The attacker used no tools. They right-clicked and read the source.
The video boils the common vibe-coding traps down to five checks that need no code. We ran our own site through all five the same day. Here is what we did and what we found.
The five checks
| Check | How we tested it |
|---|---|
| 1 · Can the database be read without logging in? | Hit the Supabase REST API directly with the public front-end key, reading and writing every table |
| 2 · Are there secrets in the source? | Downloaded every JS bundle the home page loads (13 files, ~840 KB) and grepped for sk_live, whsec_, service_role and price IDs |
| 3 · Can someone keep hammering it? | Read every API route looking for rate-limiting code |
| 4 · Add one to the number in the URL | Listed every path and endpoint that fetches by ID and checked whether it verifies ownership, not just login |
| 5 · Walk in through an incognito window | Requested the admin pages, dev tools, account page, paid pages and every endpoint with no cookies |
Results
Check 1 · Pass
All eight tables have row-level security on. Reading purchases, entitlements, user_progress, mock_attempts, page_views, route_geometry, redeem_codes and admins with the anonymous key returns an empty array every time; anonymous inserts are rejected by the database (error 42501). Every policy for signed-in users is auth.uid() = user_id, so a user can only touch their own rows. The redeem-code, analytics and route-geometry tables have no policies at all; only the server key can reach them.
Check 2 · Pass
The only key in the bundle is Supabase's publishable key, which is designed for browsers and can do exactly what RLS allows. No Stripe secret, no webhook signing secret, no service-role key, no price IDs.
One blemish: the variable name SUPABASE_SERVICE_ROLE_KEY appears in the bundle. Next.js never inlines environment variables that don't start with NEXT_PUBLIC_, so the value is undefined and nothing leaked. It does mean the module that reads the server key is imported by browser-side code, and should be split.
Check 3 · Gap
There was no rate limiting anywhere. All three public endpoints could be called without limit: the page-view beacon could be written to by any script forever; redeeming activation codes and creating Stripe Checkout sessions require login, but nothing limits a logged-in caller. Sign-in and sign-up go straight to Supabase Auth, which has its own limits, and never touch our API.
Check 4 · Pass
No URL anywhere fetches data by user ID or order number. The account page queries only the signed-in user on the server, with RLS as the second layer. The admin actions for granting features and issuing or deleting codes re-check the admins table on every call. The route-editor API validates ID format and is admin-only. The payment success page carries no session ID.
Check 5 · Pass
Without cookies: the three admin pages and the route-editor API return 404 (not 403, so they don't reveal themselves); dev pages and endpoints are 404 in production; the account page redirects to login; checkout and redeem return 401; the Stripe webhook returns 400 without a signature. Paid lessons and the 50-question exam render the paywall on the server; neither the HTML nor the RSC payload contains lesson text or questions.
How we added rate limiting
We didn't want another third-party service for this, and we didn't want a limiter outage to block payments. We ended up with two layers.
Memory layer. Each Vercel instance keeps its own Map of recent timestamps per IP. Free and instant, but not shared between instances, so a flood is only divided by the instance count. This guards the analytics beacon: excess requests are silently dropped and still get a 204, so a script can't tell it has been limited.
Database layer. A rate_limits table in Supabase plus a security definer function that does "reset if the window expired, otherwise increment" in one upsert and returns whether the limit is exceeded. Only service_role may execute it; anonymous and signed-in callers are refused. This guards redeem and checkout and counts across every instance.
insert into rate_limits (key, count, window_start)
values (p_key, 1, now())
on conflict (key) do update set
count = case when rate_limits.window_start < v_cutoff then 1
else rate_limits.count + 1 end,
window_start = case when rate_limits.window_start < v_cutoff then now()
else rate_limits.window_start end
returning count into v_count;
return v_count <= p_limit;Thresholds: the beacon at 30 per IP per minute; redeem at 10 per user and 30 per IP per ten minutes; checkout at 6 per user per ten minutes. If the function is missing or the database is unreachable, the limiter fails open. A limiter outage must never become a payment outage.
Verified on the production database after deploy: the same key with a limit of 2 returned true, true, false over three calls; calling the function with the anonymous key returned permission denied; reading the table anonymously returned nothing.
One number
From finishing the video to verified in production, the audit and the fix took about two hours. The checks themselves needed almost no code: one curl, one grep, one incognito window.
If you build with AI too
- Turning on RLS is the start. Actually read every table with the anonymous key.
- Download the JS bundles and grep them. Don't stop at the HTML.
- "Requires login" is not "is limited". Rate-limit the endpoints behind login as well.
- Block protected pages on the server. Don't ship the content and hide it with CSS.
- Design the limiter to fail open, or it becomes your most fragile single point of failure.
Source video: 小袋鼠KAKO, "AI 做完的网站，直接上线安全吗？上线前你必须做的 5 个检查".