Here are some DNS-OARC development highlights from the past couple of months. These updates are usually bi-monthly and previous ones are available on our Medium blog.
My last blog post was published on APNIC’s blog, if you missed it then please feel free to read up on how dsc-datatool and Grafana can give your DSC data a facelift.
Project: Soteria
This project was aimed at greatly improving the quality and testing of DNS-OARC’s software and tools, by using code coverage and code analysis, and was funded by The Swedish Internet Foundation.
SUCCESS!
The project was wrapped up in September after over 200 hours were spent working on it and we now have good tools and visuals in place to keep track of our code quality. For code analysis we use scan-build, part of Clang, lgtm.com and sonarcloud.io, and for code coverage we use GNU GCC’s coverage (gcov) and Python coverage. Both of the code coverage tools integrated nicely with SonarCloud for reports and visuals.
Across all projects 139 code analysis issues have been fixed, 255 false-positives have been removed and code coverage was increased from 30% to 62%.
scan-build
The Clang Static Analyzer is a source code analysis tool, part of the Clang project, which uses a collection of algorithms and techniques to analyze source code in order to automatically find bugs.
We use the command line tool scan-build to automatically run this analysis on every pull request, using Buildbot, that comes to our projects.
LGTM
LGTM is a “free for open source projects” cloud based tool that uses a variant analysis platform to automatically check code for real CVEs and vulnerabilities.
We use their GitHub App to integrate our projects and run analysis on pull requests and you can see the status of our projects.
SonarCloud
SonarCloud is another “free for open source projects” cloud based tool that we use for static code analysis. This is part of SonarSource and has been around since 2008.
While there is a GitHub App for SonarCloud, it only supports full integration with Python projects, so for this analysis we run their standalone tool sonar-scanner on a continuous basis outside of pull requests. See the status of our projects.
GNU GCC’s coverage (gcov)
Thanks to how Gcov works, by profiling the code while executing it, it was very easy to use the already existing tests as a boilerplate for coverage analysis and extend on it.
Running Gcov requires special compile options and all projects have been given the configure option --enable-gcov to enable this. This is used in conjunction with the SonarCloud analysis to generate both their report and Gcov’s code coverage report. SonarCloud also has support for reading Gcov’s reports so we also use that to present the code coverage statistics.
Python Coverage
For Python we use the more-or-less standardized Coverage.py, which been around for almost 20 years now, together with pytest.
Luckily SonarCloud can also read the XML reports produced by Coverage. So in similar fashion as with Gcov, we run this analysis in conjunction with the SonarCloud analysis to generate both reports and use its presentation of the code coverage statistics.
New releases
Due to a lot of issues found by code analysis and code coverage tests there have also been a lot of new releases, and here is a list of them with highlighted fixes:
- tinyframe v0.1.1:
tinyframe_write_control(): check for valid content field types - dnswire v0.2.0:
dnstap_decode_protobuf(): Fix setting of unknown socket family and protocol, was setting DNSTAP_MESSAGE_TYPE_ enums.
enum dnstap_message_type: Fix typo in unknown enum, now correct DNSTAP_SOCKET_FAMILY_UNKNOWN
dnswire_writer_set_bufsize(): Fix bug with changing buffer size while having something in the buffer - dsc v2.11.2:
Fix bug in asn_indexer, didn’t enable MaxMindDB after successful initiation - dsc-datatool v1.0.2:
Fix bug in DAT file parsing - dnscap v1.12.0:
Fix handling of -? option for dnscap and all plugins
plugins/rzkeychange: Add -D, dry run mode, for testing. Fix handling of -a and error on too many - packetq v1.4.3:
Add DNS resource record types SVCB and HTTPS
For full release notes please see each project’s repository and releases on GitHub.
OARC Software Chat
We recently launched our Mattermost chat platform for the DNS Operations Community and OARC Members, and I’ve created a channel on it called “OARC Software”. This channel can be used to talk about OARC’s software and its features, and for me to announce new releases. It is also a place where you can interact with others using our software and get help (as a compliment to some of our mailing lists).
https://chat.dns-oarc.net/community/channels/oarc-software
Hope to see you online there!
Cheers,
Jerry
