OMS & GDPR = Purge
Hello World,
GDPR, We all talk about it, we all work on it, we were looking in all directions for personal information collected and stored in our systems.
What about OMS?
How can we make the information collected by OMS GDPR complaint?
Well, the answer is simple and can be achieved in a few steps:
First, If possible try to avoid the collection of sensitive information in the first place.
If not we have the option to delete the information if needed as a last resort.
Filter
Let’s start by Not collecting using OMS Application Insights and Log Analytics:
In Application Insights:
- Use data filtering for sensitive information. Application Insights is used for performance and development purposes and not for sales, keep it that way.
Link to Application Insights Doc about data filtering
- Snapshot Debugger used in AI is collecting all of the information without the option to filter so just disable in a case of non-compliance.
- Custom data is collected by you and there for saving it in the database is your responsibility, but, using the next query can help you find the source:
search * | where isnotempty(customDimensions) | where timestamp > ago(1d) | project $table, timestamp, name, customDimensions
- User names/IP addresses, this information is obfuscated by default or AI is using randomly generated ids. but some information still can be saved in some cases and therefore use the next query to find it:
search client_IP != “0.0.0.0” | where timestamp > ago(1d) | summarize numNonObfuscatedIPs_24h = count() by $table
In Log Analytics:
- In the case of Log Analytics by avoiding the collection of Event Logs\Application Logs that hold personal information as much as possible.
- Same as with AI look for the IPs collected by using the next query:
search * | where * matches regex @’\b((25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)(\.|$)){4}\b’ //RegEx originally provided on https://stackoverflow.com/questions/5284147/validating-ipv4-addresses-with-regexp | summarize count() by $table
- User information can be found with a simple “search “[username goes here]“” search.
Delete
Ok, you got to the point that you need to remove sensitive information from your Log Analytics/AI, how can you do this?
To say the truth, easy, too easy
For Log you have this POST:
POST https://management.azure.com/subscriptions/
{subscriptionId}/resourceGroups/{resourceGroupName}/providers/
Microsoft.OperationalInsights/workspaces/{workspaceName}/purge
This post can be used with a query for the data to be purged
{
“table”: “Heartbeat”,
“filters”: [
{
“column”: “TimeGenerated”,
“operator”: “>”, “value”: “2019-01-01T00:00:00”
}
]
}
Same for the Insight Data but the link is deferent:
POST https://management.azure.com/subscriptions/
{subscriptionId}/resourceGroups/{resourceGroupName}/
providers/Microsoft.Insights/components/{resourceName}/purge
Be careful and have Fun.
Related Posts
About The Author
Lior
Cloud Monitoring specialist, with more than 10 years of experience in providing brode wide monitoring solutions for SMP and Enterprises. I work at CloudValley, Microsoft's partner of the year 2015&2017 and largest MSP in Israel.