Register an external system that checks your environments, give it a signing key and the checks it may answer, decide what counts as passing, and find out quickly when it stops reporting or its reports are refused.
Some controls are best checked by a machine: a cloud configuration scanner, an endpoint agent or a scheduled script. A reporting system is such a check that you run yourself. It sends ComplyTrain signed reports of what it observed, for example "examined 47 storage buckets, 47 encrypted". ComplyTrain holds no credentials for your systems and the reporting system never says whether the control passed: ComplyTrain decides that from the counts, using thresholds you set.
Product location: /qms/controls/producers. Open Reporting systems in Controls.
Task | Permission |
|---|---|
Register, enable or disable a reporting system; issue and revoke signing keys; create, enable, disable or delete bindings; set up and use Check now | Manage the estate ( |
Add the implementation the system answers, record its design review and thresholds | Manage controls ( |
Read Reporting systems, Reporting health and the reports received | View controls ( |
Organisation administrators can do all of these. Your plan sets how many implementations can be reported by an external system; see plan limits.
Product location: /qms/controls/{controlId} (fallback: /qms/controls). Open the control from the Library and choose the Where it runs tab.
A reporting system answers one or more implementations of your controls. Before you connect it, add the implementation on the control's Where it runs tab:
Choose Add.
In How it is carried out, choose Reported by an external system.
In Runs on, choose the environment the system checks, for example the production AWS account.
In How often, choose a schedule. A schedule is required, because the due date is what makes a missing report show as overdue.
Select Start it running straight away, or switch it on later with Switch on. Reports for an implementation that is not running are refused.
Nothing is created for anyone to do. When a check falls due, ComplyTrain simply expects a report. See where and how a control operates.
Product location: /qms/controls/producers. Open Reporting systems in Controls.
Open Reporting systems in Controls.
Choose Register a system.
Complete the fields and choose Save.
Field | What to enter | Limits |
|---|---|---|
Code | The identifier the system sends with every report, for example | Required. Letters, digits, dot, colon, underscore or hyphen; 2 to 100 characters; unique. Cannot be changed later, because renaming it would break the running integration. |
Name | What the system is, for example "Cloud posture scanner (production)". | Required, up to 300 characters. |
Who to contact | The email address of whoever runs the system. | Optional, up to 320 characters. Shown on Reporting health so someone knows whom to chase. |
The system appears in the list with the state Never reported. Choose Manage to open its settings.
Product location: /qms/controls/producers. Open Reporting systems in Controls.
The system signs every report with a key, and ComplyTrain checks the signature. The key never travels with the report.
In the system's Signing keys section, choose Issue a signing key.
In Your signing key, choose Copy and paste the key into the reporting system's secure configuration.
Choose I have saved it.
The key is shown only this once; nobody, including support, can look it up again. If it is lost, issue a new key and revoke the old one.
A system can hold two keys at once, so it can move to a new key on its own release schedule. To rotate: issue the second key, install it, wait until its Last used time shows it is signing, then choose Revoke on the old key and confirm Revoke key. A revoked key stops working immediately. While both slots are in use, Issue a signing key is unavailable.
Product location: /qms/controls/producers. Open Reporting systems in Controls.
A binding lets the system report on one implementation, and nothing else. The bindings are listed under What it may report on, with Control, Environment, Binding reference and Reporting.
In the system's settings, go to What it may report on.
Add a binding and choose the implementation. Only implementations carried out by Reported by an external system can be chosen.
Give the Binding reference to whoever configures the reporting system. It identifies the check the system is answering.
An implementation can have only one active binding. To move a check to a different reporting system, disable the old binding, then bind the new system. A binding that has never received a report can be deleted; once it has reported, disable it instead so its history stays. Disabled bindings show Disabled.
Product location: /qms/controls/producers. Open Reporting systems in Controls.
A reporting system always finds its work by asking ComplyTrain what is due, which works even from inside a network ComplyTrain cannot reach. Optionally, ComplyTrain can also call the system to say there is something to check now.
Setting | Options and rules |
|---|---|
Can we tell it to check now? | No — it asks us when it is ready (default) or Yes — call this address. |
Address to call | For Yes — call this address only. A public HTTPS address; addresses inside private networks are refused. Each call is signed with the system's own key. |
Choose Save. If calls fail, the section shows Failing: 3 consecutive attempts and when the address was Last reached.
To ask for a result now, choose Check now on a binding. The result says how many checks were notified straight away, how many will be picked up at the system's next check and how many could not be reached. For a system that only asks, the page reminds you it will pick the work up on its next check. A successful notice does not mean a result has arrived; watch Reporting health.
Product location: /qms/controls/{controlId} (fallback: /qms/controls). Open the control, choose the Where it runs tab and choose Review design on the implementation.
A report says how many items passed the check, not whether the check was worth running. Until a person reviews its design, the control's effectiveness stays Not yet known, however many good results arrive.
On the control's Where it runs tab, choose Review design on the implementation.
Under Record a review, answer Is this check well designed? with Yes — it would catch what this control is for or No — it would miss the thing this control is for.
Write What did you conclude?; it is required when you mark the check deficient.
Choose Record the review.
If the implementation's configuration or frequency changes later, the headline changes to This check has changed since it was reviewed and the control returns to Not yet known until someone reviews it again. Withdraw the review removes your review.
Under What counts as passing, set the thresholds and choose Save thresholds:
Field | Meaning | Default |
|---|---|---|
Passing percentage | Share of examined items that must be satisfactory for the result to read fully effective. | 100%, so every item must pass. |
Partially effective from | At or above this share, a result reads partially effective rather than ineffective. | 1%, so only a complete failure reads ineffective. |
Leave a field empty to inherit the value shown as Currently inherited. When you save, results already received are re-evaluated. Any result whose verdict changes is kept and marked as replaced, with the reason.
Product location: /qms/controls/producers. Open Reporting systems in Controls.
On a binding, choose View what it sent. Reports received lists every report, newest first, with Received, Outcome (Accepted or Refused), What it found, for example 44 of 47 passed, and Period.
Each accepted report is recorded as a test result for the check that was due, and counts toward the control's effectiveness like any other test. A refused report records nothing but its reason. A report with a wrong or missing signature is not recorded at all, so it looks like silence.
Product location: /qms/controls/reporting-health. Open Reporting health in Controls.
Open Reporting health in Controls; its Reporting systems link returns to the list of systems. Automated checks counts every active binding; the other tiles count them by state. Choose a tile to filter the table and read advice for that state.
State | Tile | Meaning | Who should act |
|---|---|---|---|
Reporting | Reporting normally | Reports are arriving within the expected interval. | Nobody. |
Never reported | Never reported | Nothing has been accepted since the binding was created. Usually the setup: wrong key, wrong binding reference, or the system was never deployed. | Whoever set up the system. |
Stopped | Stopped reporting | It reported before but has been silent for more than twice its schedule, for example more than 2 days for a daily check or 62 days for a monthly one. | Whoever runs the system, or the owner of what it watches. |
Being refused | Being refused | Reports are arriving and being refused. The latest reason is shown under the state. | Whoever runs the system. |
The table shows Control, Reporting system with its contact, State, Last reported or Never, and Waiting: how many due checks are still waiting for a report. An implementation's missing report also shows as overdue on the Due board.
The Automated Control Reporting Health report in Reporting gives the same states as a report you can export and schedule.
Product location: /qms/controls/producers. Open Reporting systems in Controls.
What you see | Check |
|---|---|
Never reported, nothing under Reports received | The system has the current signing key, the right code and the right binding reference, and it is actually running. |
Being refused | The refusal reason. Common causes: the implementation is not running, the environment has been decommissioned, or the report's period or counts are invalid. |
Stopped | The system itself, and whether it can still reach ComplyTrain. Contact the person in Who to contact. |
Check now reports failures | The Address to call is reachable from the internet and accepts the signed call. |
Product location: /qms/controls/producers. Open Reporting systems in Controls.
Choose Disable this system to stop every binding of that system at once, for example when it is decommissioned or a key may be compromised. Enable this system turns it back on. Disabling keeps the system, its bindings and every report it sent, because those reports are evidence of what was answered.
Product location: /qms/controls/producers. Open Reporting systems in Controls.
Your control "Encrypt stored customer data" must hold on the production AWS account.
On the control's Where it runs tab, add an implementation Reported by an external system, running on the production AWS account, Every day, switched on.
Register the system cspm-prod, "Cloud posture scanner (production)", with the platform team's address.
Issue a signing key and hand it to the platform team through your password manager.
Bind cspm-prod to the new implementation and send the Binding reference to the platform team.
Choose Review design, record Yes — it would catch what this control is for ("scans every bucket, including new ones"), and set Passing percentage to 100.
Next morning, Reporting health shows the binding as Reporting, and View what it sent shows 47 of 47 passed. The control's Tests tab shows the result.
Weeks later the binding moves to Stopped. The contact finds the scanner's credentials expired, fixes them, and reports resume.
