189 lines
4.5 KiB
Markdown
189 lines
4.5 KiB
Markdown
# Database Anonymizer
|
|
|
|
**Database Anonymizer** is a tool written in GO that allows **anonymizing or deleting data from a MySQL or PostgreSQL database**.
|
|
|
|
It addresses various use cases such as **providing developers with an anonymized copy of a database** or **satisfying the need to anonymize or delete data in accordance with GDPR** (General Data Protection Regulation), depending on the retention periods defined in the processing register.
|
|
|
|
The project includes a vast array of fakers. It also enables data generation via Twig-written templates. You can specify precise rules for each table or global rules applied to all tables in your configuration.
|
|
|
|
## Usage
|
|
|
|
### Configuration
|
|
|
|
The configuration is written in YAML. Here's a complete example:
|
|
|
|
```
|
|
rules:
|
|
columns:
|
|
phone: phone_e164number
|
|
generators:
|
|
person_name: [display_name]
|
|
actions:
|
|
- table: user
|
|
virtual_columns:
|
|
domain: internet_domain
|
|
columns:
|
|
firstname: person_firstname
|
|
lastname: person_lastname
|
|
email: "{{ (firstname ~ '.' ~ lastname ~ '@' ~ domain)|lower }}"
|
|
- table: company
|
|
columns:
|
|
name: company_name
|
|
- table: access_log
|
|
query: 'select * from access_log where date < (NOW() - INTERVAL 6 MONTH)'
|
|
delete: true
|
|
- table: user_ip
|
|
primary_key: [user_id, ip_id]
|
|
delete: true
|
|
```
|
|
|
|
### Run
|
|
|
|
To display help, use `-h`:
|
|
|
|
```
|
|
database-anonymizer -h
|
|
```
|
|
|
|
Here are examples for MySQL and PostgreSQL:
|
|
|
|
```
|
|
database-anonymizer --dsn "mysql://username:password@tcp(db_host)/db_name" --schema ./schema.yaml
|
|
database-anonymizer --dsn "postgres://postgres:postgres@localhost:5432/test" --schema ./schema.yaml
|
|
```
|
|
|
|
### List of fakers
|
|
|
|
#### Special fakers
|
|
|
|
- `"null"`: set `null`
|
|
- `""` or `"_"`: do nothing
|
|
|
|
#### Others
|
|
|
|
- `"address"`
|
|
- `"address_buildingnumber"`
|
|
- `"address_city"`
|
|
- `"address_cityprefix"`
|
|
- `"address_citysuffix"`
|
|
- `"address_country"`
|
|
- `"address_countryabbr"`
|
|
- `"address_countrycode"`
|
|
- `"address_latitude"`
|
|
- `"address_longitude"`
|
|
- `"address_postcode"`
|
|
- `"address_secondaryaddress"`
|
|
- `"address_state"`
|
|
- `"address_stateabbr"`
|
|
- `"address_streetaddress"`
|
|
- `"address_streetname"`
|
|
- `"address_streetsuffix"`
|
|
- `"app_name"`
|
|
- `"app_version"`
|
|
- `"beer_alcohol"`
|
|
- `"beer_blg"`
|
|
- `"beer_hop"`
|
|
- `"beer_ibu"`
|
|
- `"beer_malt"`
|
|
- `"beer_name"`
|
|
- `"beer_style"`
|
|
- `"blood_name"`
|
|
- `"boolean_bool"`
|
|
- `"car_category"`
|
|
- `"car_fueltype"`
|
|
- `"car_maker"`
|
|
- `"car_model"`
|
|
- `"car_plate"`
|
|
- `"car_transmissiongear"`
|
|
- `"color_css"`
|
|
- `"color_colorname"`
|
|
- `"color_hex"`
|
|
- `"color_rgb"`
|
|
- `"color_safecolorname"`
|
|
- `"company_bs"`
|
|
- `"company_catchphrase"`
|
|
- `"company_ein"`
|
|
- `"company_jobtitle"`
|
|
- `"company_name"`
|
|
- `"company_suffix"`
|
|
- `"crypto_bech32address"`
|
|
- `"crypto_bitcoinaddress"`
|
|
- `"crypto_etheriumaddress"`
|
|
- `"crypto_p2pkhaddress"`
|
|
- `"crypto_p2shaddress"`
|
|
- `"currency_code"`
|
|
- `"currency_country"`
|
|
- `"currency_currency"`
|
|
- `"currency_number"`
|
|
- `"emoji_emoji"`
|
|
- `"emoji_emojicode"`
|
|
- `"file_extension"`
|
|
- `"file_filenamewithextension"`
|
|
- `"food_fruit"`
|
|
- `"food_vegetable"`
|
|
- `"gamer_tag"`
|
|
- `"gender_abbr"`
|
|
- `"gender_name"`
|
|
- `"genre_name"`
|
|
- `"internet_companyemail"`
|
|
- `"internet_domain"`
|
|
- `"internet_email"`
|
|
- `"internet_freeemail"`
|
|
- `"internet_freeemaildomain"`
|
|
- `"internet_httpmethod"`
|
|
- `"internet_ipv4"`
|
|
- `"internet_ipv6"`
|
|
- `"internet_localipv4"`
|
|
- `"internet_macaddress"`
|
|
- `"internet_password"`
|
|
- `"internet_query"`
|
|
- `"internet_safeemail"`
|
|
- `"internet_slug"`
|
|
- `"internet_statuscode"`
|
|
- `"internet_statuscodemessage"`
|
|
- `"internet_statuscodewithmessage"`
|
|
- `"internet_tld"`
|
|
- `"internet_url"`
|
|
- `"internet_user"`
|
|
- `"language_language"`
|
|
- `"language_languageabbr"`
|
|
- `"language_programminglanguage"`
|
|
- `"mimetype_mimetype"`
|
|
- `"music_genre"`
|
|
- `"music_name"`
|
|
- `"payment_creditcardexpirationdatestring"`
|
|
- `"payment_creditcardnumber"`
|
|
- `"payment_creditcardtype"`
|
|
- `"person_firstname"`
|
|
- `"person_firstnamefemale"`
|
|
- `"person_firstnamemale"`
|
|
- `"person_gender"`
|
|
- `"person_lastname"`
|
|
- `"person_name"`
|
|
- `"person_namefemale"`
|
|
- `"person_namemale"`
|
|
- `"person_ssn"`
|
|
- `"person_suffix"`
|
|
- `"person_title"`
|
|
- `"pet_cat"`
|
|
- `"pet_dog"`
|
|
- `"pet_name"`
|
|
- `"phone_areacode"`
|
|
- `"phone_e164number"`
|
|
- `"phone_exchangecode"`
|
|
- `"phone_number"`
|
|
- `"phone_tollfreeareacode"`
|
|
- `"phone_toolfreenumber"`
|
|
- `"time_ampm"`
|
|
- `"time_century"`
|
|
- `"time_dayofmonth"`
|
|
- `"time_monthname"`
|
|
- `"time_timezone"`
|
|
- `"time_year"`
|
|
- `"useragent_chrome"`
|
|
- `"useragent_firefox"`
|
|
- `"useragent_internetexplorer"`
|
|
- `"useragent_opera"`
|
|
- `"useragent_safari"`
|
|
- `"useragent_useragent"`
|