Datagrid Documentation

References

Installation and Configuration Guides

Additional Resources

Release Notes - Datagrid VCTR BETA Program

2016-12-23

Datagrid Plesk Extension 2.1-0

  • added support for getting "BuyNow" URL from Plesk API
  • added changes and description metadata files

2016-12-02

Service 1.7

  • determine availability of vulnerability fixes for CentOS separately from Red Hat, correcting for fixes that are available in Red Hat (RHEL) but not yet available for CentOS (e.g., CVE-2016-6662 affecting mysql and mariadb packages).

No action is required for Datagrid clients and API users; Plesk extension users will automatically start receiving the correct availability data immediately. As soon as CentOS fixes become available, the they will show up as available in the Plesk extension and, if auto-fix is enabled (VCTR Pro only), the fixes will be applied.

Note that CentOS and all other major Linux distros distribute their packages through multiple repository mirrors and these mirrors take some time - typically well under 24 hours - to synchronize. As a result, once a fix becomes available, it may take several hours for it to become available for installation. When automatic vulnerability fixing is enabled, the VCTR Pro extension will automatically retry so that once the updated packages do become available on your mirror, they will be installed.

2016-11-08

Service 1.6

  • eliminate false positives for old kernel-devel packages on CentOS/Redhat: vulnerabilities will be reported only against the kernel-devel package that matches the version and release of the currently running kernel.

No action is required for the Datagrid clients and API.

2016-10-31

Service 1.5

Improved security information extraction for Ubuntu and CentOS/Redhat:

  • upgraded library for parsing Ubuntu security information to latest format
  • updated RedHat security info parsers to use newly available source package epoch data
  • fixed RedHat security info parsers to always find the correct CVSS metrics

No action is required for the Datagrid clients and API.

2016-10-03

Datagrid Plesk Extension 2.0-0

  • added Pro version:
    • UI and functionality to fix vulnerabilities through package updates
    • auto-update to fix critical, or all, vulnerabilities on a daily basis
    • update history tab
    • dashboard and email notifications

2016-07-15

Datagrid Plesk Extension 1.8-0

  • added diagnostic telemetry on lifecycle events
  • fixed CentOS 5 install failure

2016-07-10

Datagrid Plesk Extension 1.7-0

  • fixed daily telemetry report on CentOS/RedHat systems
  • added diagnostic telemetry on Datagrid client configuration

2016-06-15

Datagrid Plesk Extension 1.6-0

  • forward compatibility change to pm_ApiCli call for Plesk Onyx

2016-06-10

Datagrid Plesk Extension 1.5-0

  • improved support for UI themes

2016-06-08

Service 1.4

Updated the RHEL 6 and CentOS 6 vulnerability sources with a new repository path following the Red Hat Software Collections (SCL) repository change and the corresponding CentOS 6 SCL change.

No action is required for the Datagrid clients and API.

Note: If you use the SCL yum repository, you may need to change the repository name so that you will be able to update packages. See this CentOS bug and this CentOS forum post for more information.

Datagrid Plesk Extension 1.4-0

  • added support for Plesk 12.0; now supporting Plesk 12.0, Plesk 12.5 and the upcoming Plesk Onyx
  • changed vulnerability links to point to vendor-specific sites rather than to the generic NIST descriptions
  • improved handling of smaller screen sizes (mobile)
  • improved error handling on unsupported OS distros

2016-05-25 - Plesk Extension

Datagrid Plesk Extension 1.2-0

2016-05-20 - All

Notable Changes

  • Introduced a new formula for calculating the severity score of vulnerabilities, so that critical vulnerabilities that require immediate action for most systems are easier to identify (e.g., severity 10.0)
  • Added support for Docker containers running on CoreOS hosts
  • Added a new option for tracking multiple generations of (immutable) containers as a single system by matching container's name

Backward Compatibility

The new formula for calculating severity of vulnerabilities will impact existing pre-canned queries that specify minimum or maximum severity using the vuln.severity filter. Please adjust the severity level to match your intent (see details on the new severity levels). Note that the range and data type remain the same (range 0.0..10.0 as a fixed-point decimal)

Service 1.3

  • Introduced a new formula for calculating the severity score of vulnerabilities, so that critical vulnerabilities that require immediate action for most systems are easier to identify. The severity range remains 0.0 to 10.0 and the value is defined as follows:
Severity Definition Recommended Action
10.0 Critical vulnerabilities with severe impact (e.g., gaining root access or crash), typically exploitable remotely, relatively easy to exploit and having known active exploits Most externally visible servers should be fixed immediately, even if they are not hosting mission critical apps or apps storing personal information
9.0 Critical vulnerabilities with severe impact and known active exploits; unlike level 10, these are harder to exploit on servers (e.g., man-in-the-middle attacks) Servers in medium sensitivity environments will also want to fix them right away, while most others may schedule the update as part of a regular maintenance process
8.0 Critical vulnerabilities that are either hard to exploit or unlikely to be affect most typical server installations (e.g., Java vulns requiring custom code to exploit) Highly sensitive apps and/or apps executing subscriber-uploaded code (e.g., PaaS) may need to be updated immediately; most others can be updated as part of a regular maintenance process
6.0-7.0 not defined n/a
1.0-5.0 All other vulnerabilities; the severity value is based on the NVD CVSS base score and vendor-provided severity levels, with 5.0 being the highest risk/impact and 1.0 being the lowest. Impact and vulnerability should be evaluated to determine whether immediate action is required; for most systems, updates/fixes can be applied in regular scheduled intervals batching multiple updates together
0.0 Unknown severity level, e.g., for new and not-yet-published vulnerabilities Review/find more information to decide on update action. As we compile information from multiple sources, in many cases we can provide more info and severity eval even before details are published in the NVD databse.
  • Added a new option for tracking multiple generations of (immutable) containers as a single system by matching container's name. See the telemetry client section below for notes on how to enable this option

Telemetry clients: ansible-dgri-modules 1.23-1, dgri-report 1.23-1

  • Added a systemd file that can be used to install/upgrade system on CoreOS
  • Added a new option for tracking multiple generations of containers as a single system by matching container's name. This behavior can be enabled on a host by host basis in the standalone client.

    • For the standalone client, use advoptions=dgri_id_by_name on the dgri-report-setup command line or add a advoptions=dgri_id_by_name line in the /etc/dgri-report.conf file.
    • For the ansible client, add advoptions : 'dgri_id_by_name' line in the /etc/ansible-dgri-modules.conf.
    • When this option is provided, the system ID of containers will be constructed as a concatenation of the host's system ID and the container name, joined with a double underscore (e.g., i-b3841311__nginx) instead of using the full container ID. This allows containers with the same name on the same host to be treated as if they were updates of the same system, even if the containers are fully replaced (e.g., when using the immutable infrastructure approach)
  • Added a new advanced option, dgri_allow_pkg_fail, that allows for system telemetry data to be sent even if we fail to collect list of installed packages. This is useful for OS distros without a package manager (e.g., CoreOS) or those with not-yet-supported package manager (e.g., Alpine Linux).

  • fixed bugs in dgri-report-setup

CLI client: dgri-query 1.15-1

  • Added a command line option to specify custom http headers, see dgri-query --help

Note: The agate component version remains unchanged, 1.0.4

2016-05-02 - Account Dashboard

New component: Account Dashboard

The account dashboard is an open source web app that provides at-a-glance information about your infrastructure, vulnerability status and actionable items to reduce your security risk. It is packaged as a Docker container to make it easy to deploy and maintain.

Example Dashboard

The account dashboard is BSD-licensed and available at the Datagrid repository on Github. It can be used in two ways:

  1. As-is: just grab the latest version and execute the command in the README, or
  2. Fork it and modify it to fit your needs, whether they are to get different Datagrid data or integrate it within your existing dashboards and monitoring systems. If you make any changes you think will be useful to other Datagrid users, please send us a pull request.

Github link

2016-04-28 - Zapier app notifications

Datagrid VCTR can now send notifications when:

  • new systems are added, or
  • when new vulnerabilities are detected that affect existing systems, or
  • vulnerabilities affecting existing systems are updated (e.g., severity change)

Using Zapier, these notifications can trigger a variety of signals: send e-mail, SMS, Slack/Hipchat, open a ticket, etc., as well as call a REST webhook.

The Datagrid VCTR app in Zapier is in beta; you can add it to your Zapier account by visiting the Datagrid Preview app at Zapier.

For a step-by-step guide how to set up notifications, see this guide

2016-04-27 - Service

Service 1.2

  • Switched to the updated vulnerability database location for the newly released Ubuntu 16.04 LTS (xenial)
  • Improved intepretation of vulnerability data for Ubuntu vulnerabilities ("pending" state)
  • Improved handling of multiple breaks/fixes of the same vulnerability

2016-04-16 - All

Notable Changes

  • Added support for vulnerabilities for Ubuntu 16.04 LTS
  • Removed support for vulnerabilities for Debian 6 due to end of life
  • Added tracking of vulnerability's last changed date/time, allowing queries for "recently changed vulnerabilities"
  • Standalone client now installs to run as a non-root user by default (root installation also supported)
  • Improved config evaluation speed

Backward Compatibility

Service 1.1

  • Added support for vulnerability reporting for Ubuntu 16.04
  • Removed support for vulnerability reporting for Debian 6 due to end of life
  • Added a changed info field and filter for vulnerabilities, allowing queries for recently changed vulnerabilities. See examples using the vuln.changed filter in the API Reference and/or in the CLI help. A changed vulnerability is one that affects at least one of your systems AND:
    • is newly reported, or
    • has a fix becoming available, or
    • has its severity changed (e.g., due to re-evaluated impact or ease of exploit), or
    • has its description changed (and therefore may need to be re-evaluated by system administrators)

Telemetry clients: dgri-report 1.20-1, ansible-dgri-modules 1.20-1

  • Standalone telemetry client dgri-report now installs to run as non-root user by default (see updated installation instructions)
  • Added automatic config change check on reboot (standalone client only)
  • Updated state tracking so that changes reporting will be retried periodically if the Datagrid telemetry API is not reachable

dgri-query 1.14-1

  • Added display for the new changed field for vulnerabilities
  • Added examples of using the new vuln.changed filter

Note: the anonymizing gateway agate remains unchanged, version 1.04

2016-03-24 - All

Notable Changes

Backward compatibility

  • dgri_allow_dgri_id_create advanced option is no longer needed for most physical servers and VMs. Unless you use paravirtual VMs, you can remove it from the ansible config file and verify that newly created VMs have unique system IDs. Contact Datagrid support for more information

  • Prior to upgrading the Ansible client, if you are not using the anonymizing gateway, add the following line in /etc/ansible-dgri-modules.conf:

feed_url            : https://feed.datagridsys.com
  • Queries via the API and CLI tool now include only systems that have reported recently (last 3 days); data for older systems can be retrieved either by system ID or by using explicit system.last_signal filter. To include all systems in a query, including those that have stopped reporting, use the following filter: system.last_signal=min:0.

Service 1.0

  • added automatic aging-out of servers that have stopped reporting
  • support for docker containers
  • support for Debian Linux vulnerabilities tracking and reporting
  • support for Red Hat Enterprise Linux vulnerabilities tracking and reporting
  • improved vulnerability support for CentOS
  • improved system ID detection
  • API: added parent_id system field and system.parent_id filter
  • API: added enforcement of filter operators for each filter field
  • API: improved HTTP info in error responses
  • bugfixes for edge cases

dgri-report-1.18-1 - standalone client

ansible-dgri-modules-1.18-1 - ansible client

  • feed_url no longer has default; upgrades require config file change
  • added support for docker containers
  • improved system ID detection
  • improved support for hostname - if fqdn cannot be retrieved, send hostname instead
  • improved compatibility with older OS distros (Python 2.4/2.6)
  • bugfix - improved support for unusual characters in rpm info fields
  • bugfix - added support for empty Debian/Ubuntu repository lists

dgri-query-1.13-1

  • added support for Docker containers ('Parent ID' field)
  • fixed help to correctly reflect supported filter operators
  • bugfixes for edge cases

agate-1.0.4

  • better support for generating self-signed SSL certificates
  • added a new command, gencert, to make a new certificate without using --autokey
  • disabled support for SSLv3/v3 to improve security
  • documented support for https proxies
  • bugfixes for edge cases

2016-03-05 - Gateway

agate-1.0.3

  • fixed ssl failure on non-ssl http request
  • added support for ip_addr filters (inc. wildcard)
  • optimized wildcard queries with multiple system filters
  • fixed support for 'is' filter operator

2016-02-22 - All

  • Service: improved support for CentOS vulnerability identification
  • All: improved support for telemetry infomation with non-ASCII characters
Component Version
Service 0.8
Gateway 1.0.2
Telemetry 1.14-1
CLI 1.8-2

2016-02-20 - Gateway

agate-1.0.1

  • Initial release of the anonymization gateway on dockerhub

0.7 (beta) - All

  • Service supports new API (v3)
  • Service updated to support Ubuntu
  • Telemetry updated to support Ubuntu
  • CLI supports the new API (v3)

0.6 (beta) - All

0.5 (beta) - All

0.4 (beta) - All

  • This is the initial release