consent-management

Consent Management API (POC)
Product name Team Owner Tech lead Some link Lifecycle stage Tags
My product ThatTeam Cassie Cash Someguy Someguyson https://company.com/asdf Production backend

This is a basic POC for handling consents in Polestar. This project contains everything to deploy a full serverless stack into AWS.

It supports four key use-cases:

  • Command: Document new consent
  • Command: Document updated consent
  • Query: Get consent
  • Query: Get purposes

Quickstart

TODO Write an easy-to-follow guide or onboarding material for new users to help them quickly do something with the software.

See CONTRIBUTING.md for all coding standards and other questions you might have if you will start working on this product.

Prerequisites
  • AWS account with enough credentials to handle services like Lambda, S3, API Gateway, SQS, Secrets Manager
  • Recent Node (such as 16 or newer) installed
  • All commands expect Linux or Mac operability!

Pre-work before roll-out of application

Note that none of this is needed if the application is already up and running.

  • Create DeploymentRole role in AWS (using data in cicd-iam folder)
  • Set up OIDC provider in AWS for GitHub
  • Set up GitHub with required environment variables
  • Set up GitHub with permissions and repo settings

Configurations to edit

TODO Configurations that are required and/or can be edited, including for initial setup, if necessary.

Endpoints
  • Development endpoint: Link
  • Test endpoint: Link
  • Production endpoint: Link

Commands

Install

Run npm install or yarn.

Test

Run npm test or yarn test.

Run local environment

Run npm start or yarn start.

Deploy

NOTE: You will first have to deploy the stateful infrastructure (DynamoDB and SQS queues) before deploying the stateless infrastructure (API Gateway, Lambda functions, etc.).

Stateful dev environment

Run npm run deploy:infra or yarn deploy:infra.

Stateful test environment

Run npm run deploy:infra:test or yarn deploy:infra:test.

Stateful prod environment

Run npm run deploy:infra:prod or yarn deploy:infra:prod.

Stateless dev environment

Run npm run deploy or yarn deploy.

Stateless test environment

Run npm run deploy:test or yarn deploy:test.

Stateless prod environment

Run npm run deploy:prod or yarn deploy:prod.

Remove ("tear down") infrastructure

dev environment

Run npm run teardown or yarn teardown.

test environment

Run npm run teardown:test or yarn teardown:test.

prod environment

Run npm run teardown:prod or yarn teardown:prod.

Online documentation

Our online documentation can be found at:

  • Website
  • API docs
  • Confluence (add link)

Source code

Source code for our repositories can be found at:

  • Location 1 (add link)
  • Location 2 (add link)

Domain modelling

2.1: Domain modelling has been performed and documented.

Diagrams

Please see the diagrams folder.

Application code diagram

Application code diagram

Infrastructure diagram

Infrastructure diagram

Architectural Decision Records

We follow the Michael Nygard model for Architectural Decision Records. Please see the adr folder.

API contracts, interfaces and event catalog

Visit the docs on Bump or see the api folder.

Solution

API request validation

Request validation is done on AWS API Gateway using JSON Schema Draft 4 format. If you get unexpected behavior when calling it, and immediately receiving status code 400, then ensure that you are sending requests as expected. See the api folder for the files.

Multi-environment setup

We use optional headers to assign the correct usage of stages ("environments") and possible versions of the software.

Not using headers will default to the production mode.

Version

Version is for addressing evolutionary, possibly breaking changes.

  • X-Client-Version: The version of the software to run, for example 1 or 2
    • Defaults to 1 or whatever the software is set to accept as the current "steady normal state"

Stage

Stage is similar to the "environment" concept, however we don't separate the "hardware stack" based on its test/QA/dev/staging/prod status - instead we use a single hardware stack with multiple dynamically addressable stages.

  • X-Client-Stage: The stage of the software to run, for example prod or my-feature-demo
    • Defaults to prod

Example payloads

DocumentNewConsent

Endpoint to use to document a consent when you have no SalesforceID availible. Will create and return a salesforceID and document the consent.

Input

 
POST {{BASE_URL}}/DocumentNewConsent

{
  "firstName": "Test",
  "lastName": "Testesson",
  "phoneNumber": "1234",
  "emailAddress": "test@company.com",
  "purposes": [
    {
      "Id": "c894a173-0b89-449a-be9d-49bb0685d921",
      "TransactionType": "CONFIRMED | NOTGIVEN | WITHDRAWN"
    }
  ],
  "collectionPointId": "8a6c6c59-57a8-443f-88ca-5d712122a212",
  "market": "SE",
  "language": "sv-se"
}

Output

 
{
  "salesforceId": "0013H00000Waw0TQAR"
}

DocumentUpdatedConsent

Endpoint to use to document a consent when you already have SalesforceID availible. Will document the consent on the given SalesforceId

PATCH {{BASE_URL}}/DocumentUpdatedConsent

{
  "salesforceId": "0013H00000Waw0TQAR",
  "collectionPointId": "8a6c6c59-57a8-443f-88ca-5d712122a212",
  "purposes": [
    {
      "Id": "c894a173-0b89-449a-be9d-49bb0685d921",
      "TransactionType": "CONFIRMED | NOTGIVEN | WITHDRAWN"
    }
  ],
  "market": "SE",
  "language": "sv-se"
}

Output

 
204 No content

GetConsent

Gets the Purposes for the given Collection point with data wheater the data subject has consented or not to them.

GET {{BASE_URL}}/GetConsent?salesforceId=0013H00000Waw0TQAR&collectionPointId=8a6c6c59-57a8-443f-88ca-5d712122a212&market=dk&language=da-dk

Output

 
[
  {
    "id": "c894a173-0b89-449a-be9d-49bb0685d921",
    "label": "Consent Service Dev",
    "description": "Development Purpose for Consent Service v2",
    "status": "ACTIVE",
    "version": 4,
    "hasConsented": true
  }
]

GetPurposes

Gets the Purposes configured for the given Collection point, not including any consent information.

GET {{BASE_URL}}/GetPurposes?collectionPointId=8a6c6c59-57a8-443f-88ca-5d712122a212&market=dk&language=da-dk

Output

 
[
  {
    "id": "c894a173-0b89-449a-be9d-49bb0685d921",
    "label": "Consent Service Dev",
    "description": "Development Purpose for Consent Service v2",
    "status": "ACTIVE",
    "version": 4
  }
]

Roadmap and future activities

Platform support
  • Custom domain handling in Serverless Framework?

Docs to write
  • Finish the AsyncAPI docs
  • Write and finish the respective Markdown files
  • Write Confluence documentation as per Product Documentation Template
  • Write technical documentation in this README
    • 7.2: Each release is uniquely version numbered. -> How should we do this?
    • 7.3: There are release notes. -> Changelog?

Later tasks
  • Support single-environment with, for example, explicit env headers, X-Client-Version?
    • Load credentials and other things based on the version/stage etc
  • Custom domain
  • Set CORS domains on Authorizer function
  • Consider multiple keys (per app) if using API key-based solution?
  • Add additional, SLO-oriented alarms
  • Consider the use of DynamoDB to store events and keep a log of what's happening, also to increase resiliency (...privacy?)
    • When implementing caching: Finish database implementation since it is neither used nor has typed inputs and getting items is incompletely handled
  • Cleanup unused typings and document any unknown things
  • Exchange Honeycomb for Datadog?

Tests and validation

Specify how to run tests and describe what types of tests are conducted. Also, specify any manual validations that might be needed and how to do them.

Technical design of the solution

Write how you have designed your solution, for example in terms of technical scaling, databases, network segregation, and so on. See SDLC safeguards section 2 for more on these requirements.

Explicit or implicit non-functional requirements

Something here.

Technical scaling and performance

Something here.

Resiliency and reliability

8.1: Reliability is automatically calculated and we know if reliability is compromised.

Something here.

Access management

Something here.

Authentication and authorization

Something here.

Data retention and sensitivity

Something here. For more, see Data Inventory section above.

Cost

Something here.

Operational effectiveness

Something here.

Failure scenarios

2.10: Failure scenarios are modeled and solutions are designed.

Service Level Objectives

2.16: Service Level Objectives have been defined and published.

Observability

Logging

What information does this service log? Link to logs/log group.

Dashboards and metrics

Which dashboards exist? Is it easy to determine whether or not this microservice is working correctly by looking at the dashboard? Link to dashboards.

Key metrics
  • Authentication errors
  • Client errors
  • Server errors
  • Timeouts
  • Integration errors
  • Long latencies

Alerts
  • 10 or more auth errors for last minute
  • 10 or more 4xx class errors for last minute
  • Any 5XX class errors
  • Any timeout errors
  • Any integration failures
  • Latency over 10s more than 10 times in last 5 minutes

Currently all of these report to Mikael's Polestar email address.

Alarms that may be added later
  • SLO alerts
  • Availability under 99.9% for last 24 hours
  • SQS DLQ alert?

Decommissioning process

9.1: A decommissioning process exists.

Changelog

See CHANGELOG.md.

Bug and vulnerability reporting

We have a risk-based remediation strategy, with monthly, or more frequent, reviews. We use the CVSS framework to prioritize the order in which discovered vulnerabilities are fixed.

We use the following software tooling/products/services to aid us in our work:

  • Some Product

Please see SECURITY.md for more.

Contribution guidelines and code review process

Please see CONTRIBUTING.md.

Code of conduct

Please see CODE_OF_CONDUCT.md.

License

Please see LICENSE.md.

Settings

Member Visibility

Theme

Generated using TypeDoc