[HN Gopher] Grep one-liners as CI tasks ___________________________________________________________________ Grep one-liners as CI tasks Author : fphilipe Score : 61 points Date : 2022-01-14 20:40 UTC (2 hours ago) (HTM) web link (phili.pe) (TXT) w3m dump (phili.pe) | rtuin wrote: | Oh the power of Linux. | | I'm also using grep in CI as a simple but effectieve check on | dependency violations in some codebases. | jeffbee wrote: | grep and other shell tool one-liners also make excellent | kubernetes liveness checks. I have some kubernetes daemonsets | that don't do anything other than assert that things on the node | are as they should, via grep exit status. | zX41ZdbW wrote: | Here is my favorite: | https://github.com/ClickHouse/ClickHouse/blob/master/utils/c... | | "Too many exclamation marks" | david_allison wrote: | Android's default linting already contains a "missing | translation" lint rule which you can activate: | "MissingTranslation"[0] | | For Android specifically: Gradle and Android Studio both support | a powerful linting framework (to the level that it can provide | auto-fixes to the IDE). It's better to provide an in-editor to | guide your contributors before it hits CI, then have CI nag if | they didn't fix the in-editor warnings/errors: | | Some examples of custom lint rules[1] and the default rules which | Android Studio runs[2]: | | [0] | https://android.googlesource.com/platform/tools/base/+/32923... | | [1] https://github.com/ankidroid/Anki- | Android/tree/master/lint-r... | | [2] https://github.com/ankidroid/Anki- | Android/blob/master/lint-r... | excircul wrote: | > I personally prefer ripgrep as it is much faster than grep, but | usually that is not available on CI machines. | | I recommend git grep, which is comparable in speed to ripgrep, | since it ignores non-tracked files and searches the object | storage directly. It is also able to run in parallel. | jrockway wrote: | Don't you worry about what happens when the translation software | produces a semantically equivalent empty string, or something? | Like `<string><![CDATA[]]></string>`. An XML parser will find | that to be empty, grep will be like "ooh lots of text in there". | joatmon-snoo wrote: | Shameless plug: if you find yourself wanting more advamced custom | painting, you can try http://trunk.io, which provides you with | three different ways to write your own linters (pass/fail based | on your script's exit code, spit out a patch that should be | applied to your code, or LSP diagnostics) that can get propagated | to VSCode and CI (as well as just as a CLI tool, if that's your | preference). | | Disclaimer: I work on trunk :) | cfors wrote: | In today's world of excellent CLI tools I don't think grep is a | good choice, especially for checking irregular languages like | XML. [0] | | I use tools like `jq` [1] or `yq` [2] all the time for CI checks. | One useful check, is we have a configuration file stored as | several hundred lines of YAML. Its a nice thing to maintain a | sorted order for that, so we have a git pre-commit hook that runs | the following: | | > yq eval --inplace '.my_key|= sort' my_file.yaml | | Of course, a pre-commit hook or CI both work. There's pros and | cons of both. For our team, the pre-commit hook is a low enough | level of effort, and doesn't require a CI check for something | that executes in milliseconds. | | [0] https://stackoverflow.com/a/1732454 | | [1] https://github.com/stedolan/jq | | [2] https://github.com/mikefarah/yq | gravypod wrote: | I think this is a great example of where build systems, that | understand the need for testing, can help out. In a build system | I use quite often (Bazel) you can express this as an `sh_test()` | [0] which provides documentation about what you're attempting to | do _and_ provides you with a way to reproduce the failure | locally. You don 't have to push the code and wait for CI to fail | to find out, or debug, this error. | | Extra fun thing to do: print a message describing why the | decision was made and how to resolve the error in the failure | message of your test! | | [0] - | https://docs.bazel.build/versions/main/be/shell.html#sh_test | matmann2001 wrote: | Wouldn't this example do the incorrect thing in the event of a | grep error (exit code 2)? | eminence32 wrote: | The semgrep[1] tool seems like the logical next step, for when | you've outgrown plain ol' grep. | | [1] https://semgrep.dev/ ___________________________________________________________________ (page generated 2022-01-14 23:00 UTC)