Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

> able to have topics be so cheap

For GDPR a lot of us has to do exportable 'user activity'. Can you in theory have a topic/user ( we had like 50 million users) and publish any user activity to that topic?



Pulsar docs indicate "millions" of topics but IDK what 50 million would look like but from what I know I would be a bit nervous about it :)

It might be worth chatting with Pulsar devs on their slack community (https://apache-pulsar.herokuapp.com/).

Most commonly what I hear people doing for this is either one of two approaches (or a combination of both): - encrypt the user data and delete the key, eventually the user data will get removed - regularly compact the topic (pulsar has a compaction feature) and write in a tombstone record which will remove any user data after compaction




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: