Go to file
Simon Vieille 08b6e91bf3
ci/woodpecker/push/test Pipeline was successful Details
ci/woodpecker/push/build Pipeline was successful Details
Merge pull request 'fix typos in readme' (#1) from flashcode/database-anonymizer:fix-typos-readme into develop
Reviewed-on: #1
2024-04-07 22:47:14 +02:00
.woodpecker update ci condition for build 2024-04-02 19:30:43 +02:00
app manager all number types 2024-04-02 19:24:50 +02:00
config fix dsn when when th database is postgresql 2024-03-31 22:18:28 +02:00
data manager all number types 2024-04-02 19:24:50 +02:00
database manager all number types 2024-04-02 19:24:50 +02:00
faker manager all number types 2024-04-02 19:24:50 +02:00
logger add logger 2024-03-16 15:23:34 +01:00
tests tests: add mysql/postgres datas 2024-04-01 12:29:51 +02:00
.gitignore add gitignore config 2024-03-16 15:24:00 +01:00
CHANGELOG.md update changelog 2024-04-03 23:13:02 +02:00
LICENCE add licence 2024-04-01 18:38:03 +02:00
Makefile add makefile 2024-03-19 06:50:22 +01:00
README.fr.md manager all number types 2024-04-02 19:24:50 +02:00
README.md fix typos in readme 2024-04-06 17:18:25 +02:00
example.yml add/update readme 2024-04-01 15:31:20 +02:00
go.mod add progress bar 2024-04-01 18:22:03 +02:00
go.sum add progress bar 2024-04-01 18:22:03 +02:00
main.go add/update readme 2024-04-01 15:44:01 +02:00

README.md

Database Anonymizer

Database Anonymizer is a tool written in GO that allows anonymizing or deleting data from a MySQL or PostgreSQL database.

It addresses various use cases such as providing developers with an anonymized copy of a database or satisfying the need to anonymize or delete data in accordance with GDPR (General Data Protection Regulation), depending on the retention periods defined in the processing register.

The project includes a vast array of fakers. It also enables data generation via Twig-written templates. You can specify precise rules for each table or global rules applied to all tables in your configuration.

Usage

Configuration

The configuration is written in YAML. Here's a complete example:

rules:
  columns:
    phone: phone_e164number
  generators:
    person_name: [display_name]
  actions:
    - table: user
      virtual_columns:
        domain: internet_domain
      columns:
        firstname: person_firstname
        lastname: person_lastname
        email: "{{ (firstname ~ '.' ~ lastname ~ '@' ~ domain)|lower }}"
    - table: company
      columns:
        name: company_name
    - table: access_log
      query: 'select * from access_log where date < (NOW() - INTERVAL 6 MONTH)'
      delete: true
    - table: user_ip
      primary_key: [user_id, ip_id]
      delete: true

Run

To display help, use -h:

database-anonymizer -h

Here are examples for MySQL and PostgreSQL:

database-anonymizer --dsn "mysql://username:password@tcp(db_host)/db_name" --schema ./schema.yaml
database-anonymizer --dsn "postgres://postgres:postgres@localhost:5432/test" --schema ./schema.yaml

List of fakers

Special fakers

  • "null": set null
  • "" or "_": do nothing

Others

  • "address"
  • "address_buildingnumber"
  • "address_city"
  • "address_cityprefix"
  • "address_citysuffix"
  • "address_country"
  • "address_countryabbr"
  • "address_countrycode"
  • "address_latitude"
  • "address_longitude"
  • "address_postcode"
  • "address_secondaryaddress"
  • "address_state"
  • "address_stateabbr"
  • "address_streetaddress"
  • "address_streetname"
  • "address_streetsuffix"
  • "app_name"
  • "app_version"
  • "beer_alcohol"
  • "beer_blg"
  • "beer_hop"
  • "beer_ibu"
  • "beer_malt"
  • "beer_name"
  • "beer_style"
  • "blood_name"
  • "boolean_bool"
  • "car_category"
  • "car_fueltype"
  • "car_maker"
  • "car_model"
  • "car_plate"
  • "car_transmissiongear"
  • "color_css"
  • "color_colorname"
  • "color_hex"
  • "color_rgb"
  • "color_safecolorname"
  • "company_bs"
  • "company_catchphrase"
  • "company_ein"
  • "company_jobtitle"
  • "company_name"
  • "company_suffix"
  • "crypto_bech32address"
  • "crypto_bitcoinaddress"
  • "crypto_etheriumaddress"
  • "crypto_p2pkhaddress"
  • "crypto_p2shaddress"
  • "currency_code"
  • "currency_country"
  • "currency_currency"
  • "currency_number"
  • "emoji_emoji"
  • "emoji_emojicode"
  • "file_extension"
  • "file_filenamewithextension"
  • "food_fruit"
  • "food_vegetable"
  • "gamer_tag"
  • "gender_abbr"
  • "gender_name"
  • "genre_name"
  • "internet_companyemail"
  • "internet_domain"
  • "internet_email"
  • "internet_freeemail"
  • "internet_freeemaildomain"
  • "internet_httpmethod"
  • "internet_ipv4"
  • "internet_ipv6"
  • "internet_localipv4"
  • "internet_macaddress"
  • "internet_password"
  • "internet_query"
  • "internet_safeemail"
  • "internet_slug"
  • "internet_statuscode"
  • "internet_statuscodemessage"
  • "internet_statuscodewithmessage"
  • "internet_tld"
  • "internet_url"
  • "internet_user"
  • "language_language"
  • "language_languageabbr"
  • "language_programminglanguage"
  • "mimetype_mimetype"
  • "music_genre"
  • "music_name"
  • "payment_creditcardexpirationdatestring"
  • "payment_creditcardnumber"
  • "payment_creditcardtype"
  • "person_firstname"
  • "person_firstnamefemale"
  • "person_firstnamemale"
  • "person_gender"
  • "person_lastname"
  • "person_name"
  • "person_namefemale"
  • "person_namemale"
  • "person_ssn"
  • "person_suffix"
  • "person_title"
  • "pet_cat"
  • "pet_dog"
  • "pet_name"
  • "phone_areacode"
  • "phone_e164number"
  • "phone_exchangecode"
  • "phone_number"
  • "phone_tollfreeareacode"
  • "phone_toolfreenumber"
  • "time_ampm"
  • "time_century"
  • "time_dayofmonth"
  • "time_monthname"
  • "time_timezone"
  • "time_year"
  • "useragent_chrome"
  • "useragent_firefox"
  • "useragent_internetexplorer"
  • "useragent_opera"
  • "useragent_safari"
  • "useragent_useragent"