All notable changes to this project will be documented in this file.
The format is based on Keep a Changelog, and this project adheres to Semantic Versioning.
Types of changes
Added
for new features.Changed
for changes in existing functionality.Deprecated
for soon-to-be removed features.Removed
for now removed features.Fixed
for any bug fixes.Security
in case of vulnerabilities.
Added
masktimeline
to generate timelines from rules and constraints between dates
Fixed
load data for unique cache
Fixed
memory leak with context logs
Added
--pprof
flag to create acpu
ormemory
pprof fileAdded
new featuresxml
mask to to manipulate XML content within JSON values - the proposed syntax aims to align with existing masking conventions for ease of use
Added
new featuresfindInCSV
mask to get one or multiple csv lines which matched with Json entry value from CSV filesAdded
new flag--serve
to start a HTTP server that apply masking to JSON body of requestsAdded
new flag--buffer-size
to increase the buffer used to load files from URIFixed
some bugs with selectors
Added
new features for parsing and masking XML files using the following command:cat XMLfile | pimo xml --subscriber <parent tag name>=<mask name> > outputXMLfile
- this feature supports all level 1 elements that are not arrays
Added
new features for ff1 mask :domain
,preserve
andonError
Added
possibility to use.
in the jsonpath field of selectorsAdded
option--skip-log-file <filename>
to log skipped lines on error when using--skip-line-on-error
Fixed
local seed parameter should be consistent between different jsonpathFixed
PIMO Play handle panics and recover gracefully when panics occure
Added
randomChoiceInCSV
andhashInCSV
mask to get values from CSV filesFixed
Seed
option on randomized mask use global configuredseed
and the hash of the valueFixed
--seed
flag allow the-1
value
Added
Environment variablePIMO_STATS_TEMPLATE
to control the format of generated statsAdded
Environment variablePIMO_STATS_URL
to generate a stat file or HTTP POST
Added
flag to declare seed in command lineAdded
flag--stats
to generate a stat file or HTTP POSTAdded
flag--statsTemplate
to control the format of generated statsFixed
cache now apply once on whole masking item, instead of each mask in the block
Security
in PIMO Play : disallow usage of mask Command inmasks
propertyAdded
functions section in masking configurationAdded
possibility to use masks as template functions (most mask are available asMask<mask_name>
: MaskRandDate, MaskRegex, ...)Added
in PIMO Play: adds a tab to visualize a mermaid graph flow of the maskingFixed
preserve notInCache enum in the jsonschemaFixed
functions call from a pipe mask crash pipeline
Fixed
click on examples does not reload webpage on the PIMO Play websiteFixed
verbosity in pimo play command
Added
possibility to share URL directly from the PIMO Play websiteAdded
precrafted examples in the PIMO Play websiteAdded
loading screen when JS is loading on PIMO Play websiteAdded
loading spinner when refreshing output on PIMO Play websiteAdded
a refresh button on PIMO Play websiteFixed
jsonschema forrandomDecimal
andrandomDuration
masks
Added
seed option on all randomized masks, another way to preserve data coherenceAdded
transcode maskAdded
play command
Fixed
use fromcache after a mask which causes a change in the type of the value
Added
markov mask to generate pseudo text based on a sample textAdded
command to export maskings as a mermaid flow chartAdded
possibility to use optionpreserve: notInCache
with maskfromCache
Added
flag to mask input while a declared condition is metAdded
flag to mask input until a declared condition is metAdded
flag to declare a simple mask definition in command lineAdded
possibility to declare multiple selectors for a masking definitionFixed
pimo doesn't panic anymore with fromjson combined with pipe mask, or fromjson combined with another fromjson mask with nested selectorsFixed
maskreplacement
with nested selectorsFixed
using fromCache, the line is not deleted if the jsonpath in not present in the documentAdded
option incaches
section for use reverse cacheAdded
unixEpoch format with maskdateParser
Added
option preserve in masking configuration.Fixed
cache with maskfluxUri
.
Added
luhn mask to generate valid checksums using the Luhn algorithm.
Fixed
JSON Schema validation formasks
property (only required ifmask
is not set).
Added
possibility to use a template string withrandomChoiceInUri
maskAdded
new maskadd-transient
same asadd
but the field is not exported in the jsonline outputAdded
possibility to use a template string directly withadd
maskAdded
possibility to chain multiple masks in YAML configuration on the same jsonpath with themasks
property
Added
new maskfromjson
to convert a string to object model
Added
new masktemplate-each
to mask each value of an array with a templateFixed
pimo doesn't crash anymore when nulls values occurs in the middle of a path
Fixed
Protect masks against null values (dateparser, duration, hash, randdura, range)
Fixed
Range over array in template mask
Fixed
Pipe mask with repeat flag failed in panic (#34)Added
jsonschema command to generate schema of masking.yml fileAdded
flag to enable or disable coloring in output logs (--color [yes|no|auto])
Changed
order of keys on JSON objects will now be preserved on the output
Added
new functions intemplate
mask via sprig dependency v3 (durationRound, numerous, toRawJson, htpasswd, duration, seq, randInt, fromJson, mustFromJson, bcrypt, randBytes, dig, regexQuoteMeta, osBase, osDir, osExt, osClean, osIsAbs, and, all, addf, add1f, subf, divf, mulf, maxf, and minf, chunk, and more...) and improve others (get)Added
strutured logging with-v
and--log-json
flagsAdded
debug option--debug
, warning it's slow do not use in productionFixed
flag--skip-line-on-error
Fixed
flag--skip-field-on-error
Fixed
handling of null values by the theff1
mask
Added
ff1 mask to meet the requirement of re-identification from a single secret key.Added
pipe mask to handle jsons with complex structure (nested arrays of objects)Fixed
use same random mask on different path produce the same value.Fixed
masking values in nested arrays generate panic error.
Added
First public version released