DG AGS4 reducer tool

Reduce an AGS file by deleting selected groups and/or selected holes (LOCA).

Instructions
  • Select file to process using file picker below
  • Click button to run the first stage of the process (analyse file)
  • On the next screen, select which GROUPS and/or LOCA_ID to EXCLUDE, i.e delete
  • Click button to run the file reduction process
  • A summary of the outcome is then presented on screen (this may also be downloaded as a file)
  • Click on relevant button to download the 'reduced' file generated

Please be aware that this tool will allow you to produce a reduced file that may not constitute a valid AGS file in accordance with the AGS format rules. This is deliberate as we recognise that this may not be important for the process that you are implementing.

However, if you are using this tool to produce 'valid' AGS files for issue then you should exercise caution. For example, when selecting groups to delete you must ensure that you do not compromise parent-child relationships, or remove mandatory groups. This tool will flag up some potential validation issues, including the above, which will be presented as warnings. It does NOT include full validation. We recommend using our validator tool on the reduced file if this is important to you.

If you are having problems, or if you would just like to provide some feedback, then please do not hesitate to contact us.

Limitations

Additional technical information

The 20MB file limit is primarily dictated by the limitations of Google App Engine (GAE) Standard Environment which we are using to host these apps. This limit reflects the maximum file size we have tested for so far. Please get in touch if you need to process larger files. This is very do-able, but it will require a different implementation, e.g. running the underlying python code locally or running it on an upgraded (paid for) implementation of GAE.

Selection of LOCA_ID for exclusion is only possible if the input file has a LOCA group.

On the selection page, groups and LOCA_ID are listed in the order they appear in the input file (in LOCA for LOCA_ID).

When groups are selected for exclusion, they will be deleted in their entirety, including all header lines. During the reduction process, deletion of the groups selected for exclusion is carried out first.

Next, all records in all remaining groups with LOCA_ID selected for exclusion are identifed and deleted. Groups that do not have LOCA_ID are not touched during this process.

After the LOCA_ID removal, a further check is carried out to see if there are any groups with no DATA rows remaining, i.e. all of the LOCA_ID in those groups deleted. Any such groups found are also deleted, i.e. redundant header lines removed. Any such additional deleted groups are flagged up in the 'results'.

Finally, all records in the ABBR group that apply to any of the deleted groups are removed (we check the first part of ABBR_HDNG).

Please note that the above is the only change we make to ABBR. It is possible that some of the remaining ABBR entries may be obsolete due to deletion of LOCA_ID that used them. It is not a straightforward task to identify these records so we have decided to leave them as they are for now.

As noted at the top of the page, this tool will allow you to generate AGS files that may not be valid in accordance with the AGS rules. However, we check and show warnings if any of the following apply:

  • Mandatory groups deleted
  • Parent groups deleted, leaving orphan child groups

We do NOT check for compromised record-link data.

If there are AGS format rule validation issues in the original file, then it is likely that the same problems will propagate to the reduced file, unless the problems are so severe that they prevent the file from parsing in the first place!

However, there are a few situations where rule validation issues may be ignored/corrected during parsing. One example is that data will be parsed even if quotemarks for data items, i.e. "...", are omitted (rule 5). No warning is provided for this particular trangression. Do not assume that a file is valid AGS just because no warnings are flagged up.

If you are concerned about AGS rule validation then we recommend using our validator tool on one or both of the original and reduced files. This reducer tool does not carry out full validation and will only flag up major issues that affect the processes it uses.

Revision history

Only changes that affect the results and other signficant changes are listed here. Minor cosmetic changes may not be included.

Date Description of change
01/02/2022 First stable release