Now, let’s try a much better, O(1) control plane. DUMB AS ROCKS. Imagine if instead that the customer API pretty much edits a document directly, like a file on S3 or whatever. And imagine the servers just pull that file every few seconds, WHETHER IT CHANGED OR NOT.
This system is O(1). The whole config can change, or none of it, and the servers don’t even care. They are dumb. They just carry on. This is MUCH MUCH more robust. It never lags, it self-heals very quickly, and it’s always ready for a set of events like everyone changing everything at once. The SYSTEM DOES NOT CARE.
GDPR’s right to be forgotten means we have to be able to erase a person’s data from our systems. Event sourced systems work from an immutable log of events which makes erasure difficult. You probably want to think hard about storing data you need to delete in an immutable event log but sometimes that choice is already made and you need to make it work, so let’s dig in.
Erasing user data from current state projections
This is relatively straightforward. A RightToBeForgottenInvoked event is added to the event store for the person. All projectors that depend on personal data listen for this event and prune or scrub the appropriate data for the person from their projections.
Erasing data from the event stream itself
This case is trickier. We need to rewrite history in a way that doesn’t break things. Let’s look at an option for erasing data without rebuilding the event stream. This approach is also applicable for projections that are immutable change logs.
We can store personal data outside of events themselves in a separate storage layer. Each event instead stores a key for retrieving the data from this layer and any event consumers request the data when they need it. Given this data is personal the storage layer should probably encrypt the data at rest.
Once a RightToBeForgottenInvoked event is added to the event store all data for that person can be erased from the storage layer. All subsequent requests for data from the secure storage layer for that person’s data will return null objects rather than the actual data. This should make life easier for all consumers and avoid you null checking yourself to death all over the place.
Let’s see what this secure storage layer might look like.
Sketch of a secure storage layer
Our secure storage layer stores data that is scoped to a person and has a type (so we can return null objects). The store allows all data for a specific person to be erased.
Let’s start with two main models: a Person1 and a Data model.
Data Person
┌──────────┐ ┌───────────────┐
│ id │ ┌───>│ id │
├──────────┤ │ ├───────────────┤
│person_id │───┘ │encryption_key │
├──────────┤ ├───────────────┤
│ type │ │ is_erased │
├──────────┤ └───────────────┘
│ciphertext│
└──────────┘
The interface to the secure storage layer is outlined below.
classSecureStoragedefadd(person_id,data_id,type,data)# Find the Person model for person_id (lazily create one if needed).## Encrypt the data using the person's encryption_key and store the# ciphertext in the data table using the client supplied data_id and type.## Clients will store this data_id in an event body and use it to retrieve# the data later.enddeferase_data_for_person(person_id)# Mark the corresponding record in the person table as erased# and delete the encryption key.enddefget(data_id)person=Person.find_non_erased(person_id)ifperson# Look up the row from the data table, decrypt ciphertext using the# key on the person model, and return the data.else# Look up the row from the data table and return a null object for# that data type.endendend
Where does that leave us?
After a person has invoked their right to be forgotten all current state projections will be updated to erase that person’s data. The event store will return null objects for any events that contain data for the person which means that any event processors won’t see that data as they build their projections. It will also contain the RightToBeForgottenInvoked event for the person so consumers can handle that explicitly if required.
This could be expanded to be more general but we’ll stick with person for the purpose of this post. ↩
I removed Google Analytics from my sites1 but still wanted access to some simple request statistics on them. Turns out GoAccess gives me most of what I need by analysing my Apache access logs.
The main challenges I ran into were, working out the correct flags for GoAccess, feeding compressed and uncompressed access logs at the same time, and ignoring junk requests from internet pests.
I pulled together a bash script that handles this, the essence of which is below.
# Analyse all log files{cat /var/www/mysite/logs/access.log; zcat /var/www/mysite/logs/access.log.*.gz;} | \# Strip out junk requestsgrep-v-E'\.php|jmx-console|\.cgi|phpmyadmin|dbadmin' | \# Fire up goaccess using the correct log file format for consolidated apache logs
goaccess --log-format='%h %^[%d:%t %^] \"%r\" %s %b \"%R\" \"%u\"'--date-format='%d/%b/%Y'--time-format='%H:%M:%S'--ignore-crawlers
A) It does way more than I need. B) I still don’t understand how to use it properly. C) It needlessly collects and sends your information off to the Big G. ↩
Learning something new all the time, and staying healthy. Getting paid. Interacting with smart people. Having the chance to pass something along to others.
If you think about it this way, you might recognize that in many software teams, our limitation is not how much we can do, but how much we can know. To change a sufficiently complex system, we need more knowledge than one or two people can hold. Otherwise we are very slow, or we mess it up and the unintended effects of our change create a ton more work.
So the next time you find yourself tempted to volunteer to take over responsibility for something from someone who reports to you, pause. Instead of taking it over, ask them what they think the next steps should be. Give your feedback, and let them bring the follow-ups to you in the next appropriate touch base. You owe it to them to stop taking over their work in the name of helpfulness.
There are three types of product features, a seasoned head of product told me recently. MMRs, neutralizers, and differentiators. MMRs are minimum market requirements; basic features that every customer expects and demands. Neutralizers mitigate competitive threat. Differentiators are your startup’s competitive advantage.
…
As the product team talks to customers, they are likely to hear feedback encouraging more investment in MMRs and neutralizers. “Your product is missing this feature. We need this capability that exists in another piece of software in yours.”
…
customers rarely push vendors to further their differentiation. By definition, the unique selling proposition doesn’t exist elsewhere in the market. So advances or improvements to the differentiator may not be obvious to customers.
it’s important to build up a solid instinct for separating the technologies and products worth spending time on from the ones that will fade into obscurity after their 15 minutes of fame is over
The Tesla’s big battery in South Australia has already taken a 55 per cent share in the state’s frequency and ancillary services market, and lowered prices in that market by 90 per cent, new data has shown.
…
And the 100MW battery has achieved over 55 per cent of the FCAS revenues in South Australia. So it’s 2 per cent of the capacity in South Australia achieving 55 per cent of the revenues in South Australia.
…
Various estimates have put the cost savings to consumers from the FCAS market alone at around $35 million, just in the first four months of its operation.
That’s a pretty good bang for the buck for the estimated $50 million investment by the South Australia government. South Australia is the only state that has experienced a decline in FCAS prices over the past few months.
What follows is my personal, opinionated, idiosyncratic approach to React/Redux development, forged in half a dozen attempts to pursue test-driven development at a good pace.
Some interesting approaches in here. I like the use of more classic terms for some of the Redux concepts. Commands instead of Action Creators. Events instead of actions. Read models instead of reducers or selectors.