Skip to content

Update main to v0.1.1

Matthew K Defenderfer requested to merge rel_v0.1.1 into main

This merge request adds some minor usability improvements as well as bugfixes. Scripts used to run policies are not affected and will be updated in a separate request.

Changes:

  • removed dependence on sif for convert-to-parquet.py's environment. Instead, run-convert-to-parquet will automatically source a virtual environment stored in /data/rc/gpfs-policy with the necessary packages and Python interpreter. (Fixes #23 (closed))
  • Set the default path for split files to be in a directory with the same name as the log file being split. This directory will be in the same place as the log file. (Fixed #19 (closed))
    • The output directory can also now be specified by the user
  • Added and improved documentation for each step in the current pipeline from running a policy to converting split logs to parquet

Bugfixes:

  • tld defaults to None when the path is not in /data/user, /data/project, /home, or /scratch (Fixes #22 (closed))
Edited by Matthew K Defenderfer

Merge request reports