Every change to our API is now attacked before we release it, and attacks on the live system found no server error and no wrong access.
Every change to our API now meets an attack on its routes, and a change cannot be released until that attack finds no server error. We then attacked the live system itself, in test clinics set up for it, with 13,797 attack requests, and not one was answered with a server error. In the part that called the API as 14 kinds of user across two test clinics, no request was let through that the user’s role should refuse.
What is new in this release
The protections that came with this release:
- Every change attacked before we release itA tool sends unexpected input to our API’s routes, signed in as a clinic owner and as a lite user, against a database built the way the live one is. It covers every route except the few that end your own sessions or need our sign-in service. If any request is answered with a server error, the check fails, and the change is held: it cannot reach the live system until every check passes.
- Every number pushed past its limitBefore that attack, a separate check sends every number a read route accepts past the largest value our database can hold. None may be answered with a server error, or the check fails.
- No passwords or keys in our code’s historyEvery change is also checked by a scan of the API’s whole history for passwords and keys written into the code by mistake, beside the check of our libraries against published security advisories that was already in place.
- Steady under heavy useWe capped a cache inside our API so its memory stays within a fixed size however many different searches people run.
How we ran it
How the live attacks were set up:
- The live system, in test clinicsWe attacked the live API itself, not a copy. Every signed-in attack used accounts in test clinics we set up for it, holding test data only; the attacks changed nothing outside those clinics, and no real email was sent. Our normal request limits stayed on, so 264 requests were told to slow down, as anyone’s would be.
- First, an attack that only readsEvery number pushed past its limit, then unexpected input sent to 76 of the API’s 78 read routes, signed in as a clinic owner.
- Then 14 kinds of user, twiceNext, a walk through the API as 14 kinds of user: signed out; a clinic owner with and without two-step sign-in; advanced, basic and lite users; users whose team limits what they can do; a user on no team; a colleague in the same clinic; and four users of a second test clinic aimed at the first clinic’s records. For each request we wrote down first what that user should be allowed to do, and the walk compared every answer with it. It also looked through every answer for records from any clinic outside the two test clinics. We ran the walk twice.
- Last, changing thingsInside one test clinic, as its owner, unexpected input sent to 98 of the API’s 112 routes that add, change or delete something.
The scans
Attack requests each part sent, and how many server errors it found. Bars are to scale.
| Part | Requests | Share of the largest | Found |
|---|---|---|---|
| Changing thingsRoutes that add, change or delete, inside one test clinic, as its owner | 4,589 | 0 | |
| Reading onlyEvery number past its limit, then unexpected input to the read routes, as a clinic owner | 4,565 | 0 | |
| Second walk14 kinds of user across two test clinics | 2,337 | 0 | |
| First walkThe same 14 kinds of user, earlier the same day | 2,306 | 0 |
What it found, and what it left out
No server error in any of the 13,797 requests. In both walks, no request was let through that the user should have been refused, including the 1,270 requests in which a user of the second clinic aimed at the first clinic’s records, and no answer showed records from any clinic outside the two test clinics. A few answers differed from the walk’s expectation the other way, a refusal where an answer was expected; none let anyone in.
Left out on the live system: the routes that send real email (inviting and managing users, support tickets, the website’s forms and password resets), the routes that sign you out everywhere, changing your own password, resetting two-step sign-in, and the desktop demo’s sign-up. ReferralFinderX’s own tools, support’s console and the data hygiene engine’s routes, were checked only to be sure clinic users are refused. Demo visitors, and accounts that are held, closed, disabled or signed out by force, were not part of these live runs; they were tested on our test copy before this release.