API ipset-ng JSON (JSONBL) communication protocol

Communication in the ipset-ng package is implemented by JSON protocol.
Implementation details and encapsulation discussed on this page.


Example JSON packet from client to server:

  • Full source from client to server send:

    ipset-ng JSON client to server packet:
    {
        "id": 5111,
        "cmd": "test",

        // only "ip" or "net",
        // if operations on the table - no set
        "ip": "1.2.3.4",
        "net": "1.2.3.0/24",

        "tbl": "whitelistng",
        "type": "white",
        "ipv": "ipv4",

        // Optional secondary check IP in DNSBL server-side:
        "dnsbl":1,

        // Optional geo location command:
        "geot": "asn",
        "geor": "AS1234,AS7895,AS3216,..",

        // Optional user JavaScript command:
        // These options make the IPSETD-NG server to scan IP addresses using a JavaScript.
        // Script name and list type is a required parameter.
        "uscr": "parseReferer.js",
        "utype": "black",
        "ustr": "any text string: Referer,Host,Cookie,User-Agent or more ..",

        // Optional structure & parameters
        // to specify additional parameters ipset tables
        "hash": "hash:ip",
        "opt": {
            "family": "ipv4",
            "hashsize": 1024,
            "maxelem": 65536,
            "timeout": 12000
        }

    }


Main field:
  • id - unique ID packet is generated automatically when sending and checked upon receipt.
  • ipv - Internet Protocol prepare IP address version, valid: ipv4 | ipv6, see command numeric details
  • ip - the IP address to which the rule applies field cmd
  • net - the network addresses to which the rule applies field cmd. For network addresses are only valid commands add | del, don't use command test.
  • tbl - server-side name of the table in which you want to check or add/remove prepare IP address or network addresses. So as with appropriate instruction performs the specified command action in field cmd on the table: create | destroy | flush.
  • cmd - command to execute request, see command numeric details:
    • valid command to prepare IP address:
      • test - test IP address
      • add - add IP address or network addresses
      • del - delete IP address or network addresses
    • valid command to prepare tables with the name taken from the field tbl:
      • create - create a table
      • destroy - delete table
      • flush - clear a table
  • type - the type of operation to be performed from the field cmd, see command numeric details:
    • black - for command test, table is type 'black list'
    • white - for command test, table is type 'white list'
    • add - for only executed command add, see detail
    • del - for only executed command del, see detail
    • table - for only executed commands create | destroy | flush, see detail

This type is set automatically by the server based on the query commands from the field cmd
There is no need to define it in advance.
Optional - parameters and features:
  • hash - ipset type tables, see detail
    • can have the following meanings:
      • list:set
      • hash:net
      • hash:net,port
      • hash:net,port,net
      • hash:net,net
      • hash:net,iface
      • hash:ip
      • hash:ip,port
      • hash:ip,port,ip
      • hash:ip,port,net
      • hash:ip,mark
      • bitmap:ip
      • bitmap:port
      • bitmap:ip,mac
  • opt - structure of additional optional parameters:
    • family - ipset table Internet Protocol version, see detail
      • ipv4 - Internet Protocol version 4
      • ipv6 - Internet Protocol version 6
      • inet - alias for backwards compatibility ipv4
      • inet6 - alias for backwards compatibility ipv6
    • hashsize - the maximum size of the table, default 1024, see detail
    • maxelem - limit the maximum number of entries in the table, see detail
    • timeout - live time added IP addresses in the table, measured in seconds, see detail
  • dnsbl - server-side enable or disable secondary check in DNSBL server, value 1 or 0. See detail

This option will work provided the ipset-ng server is configured to work with an external DNSBL source.
See the configuration file /etc/ipsetd-ng.conf from section 'dnsbl'.

Used when creating a table from the command field 'cmd': create

Used when add IP address or network addresses to table.
IPSETD-NG server, if you add IP addresses specified TIMEOUT but without the support of the creation table TIMEOUT setting, server add IP address to table ignores this parameter.
Valid ipset table types to add IP address: hash:ip or bitmap:ip
WARNING: not use ip timeout where is table set created without timeout support!
ipset v6.21.1: Timeout cannot be used: set was created without timeout support
Optional - GeoLocation command and features:

IPSET-NG package default support GeoLocation from built-in IP2Location engine and MaxMind extended library. Both of these products are used at the same time, and complement each other. IPSET-NG package should be compiled with the GeoIP support, this happens automatically if the libraries in the database and geoip installed.

  • geot - valid geoip command asn | isp | country2 | country3 | city | org | region | area | weather | domain | zipc | timez | netspeed | latlng | utype or numeric value
  • geor - any wanted string in comma separator.

Geo command for using IP2Location version #24 ipv4 and ipv6 database:

  • isp | country2 | country3 | city | org | region | area | weather | domain | zipc | timez | netspeed | latlng | utype

For the utilization of GeoIP from MaxMind, you must install the database and library MaxMind.

Geo command and base name for using full commercion version MaxMind database:

  • GeoIP.dat - country2 | country3
  • GeoIPCity.dat - city | latlng | zipc
  • GeoIPISP.dat - isp
  • GeoIPASNum.dat - asn
  • GeoIPASNum.dat - asn
  • GeoIPOrg.dat - org
  • GeoIPRegion.dat - region
  • GeoIPNetSpeed.dat - netspeed
  • GeoIPDomain.dat - domain

Geo command and base name for using GeoLite version MaxMind database:

  • GeoIP.dat - country2 | country3
  • GeoLiteIPCity.dat - city | latlng | zipc
  • GeoIPISP.dat - isp
  • GeoIPASNum.dat - asn

To freely redistribute of MaxMind GeoLite noncommercial base, please collect the package ipset-ng with key: make GEO=LITE

Optional - User JavaScript command and features:
  • uscr - name of server-side user JavaScript file. Run this file for the current request.
    Script name searches the directories specified in the server configuration parameter userscript -> base .
  • utype - the type of operation expected results, valid: black | white, see command numeric details
  • ustr - prepared string to passed in JavaScrip. Output logic of the script should be based on the analysis of this string to determine the IP addresses belonging to the group lockout or not. See example JavaScript server scenario in <ipset-ng-src-dir>/userscript/.js, or see online example.


JBL - JSON Black List import JSON format:

Structure of JSON packet:

  • table - table name
  • type - ip = 1, net = 3, set = 4, 0 = table type error, skip this table
  • ipv - ipv4 = 2, ipv6 = 10
  • el - max element in table <= 4294967295
  • is - items size, for type list:set
  • hs - hash size, 1024 (?)
  • tm - timeout in seconds
  • pkt - packets
  • bt - bytes
  • ipa - IP address array

  • Example import format:


    {"table":"blackproxy0","type":1,"ipv":2,"ipa":["195.154.251.25","74.115.3.61","199.188.236.68"]}
    {"table":"blacknet0","type":3,"ipv":2,"ipa":["142.91.0.0/16","177.47.126.0/24","193.26.64.0/19"]}
    {"table":"blackip60","type":1,"ipv":10,"ipa":["2a01:238:4389:1500:3fc9:eaf6:858e:e259","2a02:29e8:770:0:3::13"]}
    {"table":"blackip40","type":1,"ipv":2,"el":4294967295,"hs":1024,"tm":68000,"ipa":["195.154.251.25","74.115.3.61","199.188.236.68"]}



Example JSON packet from server to client:

  • Full source to receive client from server sent:

    ipset-ng JSON server to client packet:
    {
        "id": 5111,
        "ret": 404
    }


Main received field:
  • id - unique ID packet sent by the client.
  • ret - returned a response to a client request:
    • 404 - host is blocked
    • 200 - host is good status or operation is table return success.
    • 500 - error value of request, server response
    • 403 - incorrect request, bad json string, e.t.c.
    • -1 - error communication or other system error


Example of different purposes client requests:

Check IP from black list table, enable secondary check in DNSBL server:


    {
        "id": 5111,
        "cmd": "test",
        "ip": "1.2.3.4",
        "tbl": "blocklist",
        "type": "black",
        "ipv": "ipv4",
        "dnsbl":1
    }


Check IP from black list table, disable secondary check in DNSBL server:


    {
        "id": 5111,
        "cmd": "test",
        "ip": "1.2.3.4",
        "tbl": "blocklist",
        "type": "black",
        "ipv": "ipv4",
        "dnsbl":0 
    }


Check IP from white list table, disable secondary check in DNSBL server:


    {
        "id": 5111,
        "cmd": "test",
        "ip": "1.2.3.4",
        "tbl": "whitelist",
        "type": "white",
        "ipv": "ipv4",
    }


Check IP from server-side user JavaScript,


    {
        "id": 5111,
        "cmd": "test",
        "ip": "1.2.3.4",
        "uscr": "test-web-ip.js",
        "utype": "black" | "white",  // only one set
        "ustr": "Mozilla/5.0 (compatible; Konqueror/4.1; OpenBSD) KHTML/4.1.4 (like Gecko)"
    }


Insert IP to table using timeout live time IP address:


    {
        "id": 5111,
        "cmd": "add",
        "ip": "1.2.3.4",
        "tbl": "blacklistng",
        "opt": {
            // WARNING: not use ip timeout where is table set created without timeout support!
            "timeout": 3600
        }
    }


Delete IP to table:


    {
        "id": 5111,
        "cmd": "del",
        "ip": "1.2.3.4",
        "tbl": "blacklistng",
    }


Create table:


    {
        "id": 5111,
        "cmd": "create",
        "tbl": "blacklistng",
        "hash": "hash:ip",
        "opt": {
            "family": "ipv4",
            "hashsize": 1024,
            "maxelem": 65536,
            "timeout": 72000
        }
    }


Delete table:


    {
        "id": 5111,
        "cmd": "destroy",
        "tbl": "blacklistng",
    }


Flush table:


    {
        "id": 5111,
        "cmd": "flush",
        "tbl": "blacklistng",
    }


Example server returned a response to a client request:

Answer from not found or not create/insert:


    {
        "id": 5111,
        "ret": 404
    }


Answer from found or success status create/insert:


    {
        "id": 5111,
        "ret": 200
    }


Answer from found server error, table not found, parameters of request not complette e.t.c.:


    {
        "id": 5111,
        "ret": 500
    }



Support command is string or numeric format:

  • cmd - command to execute request:
    • create 2
    • destroy 3
    • flush 4
    • add 9
    • del 10
    • test 11
  • ipv - Internet Protocol version:
    • ipv4 2
    • ipv6 10
  • type,utype - the type of table & operation:
    • black 1
    • white 2
    • add 3
    • del 4
    • table 5
  • geot - valid numeric geoip command:
    • asn 1001
    • isp 1002
    • country2 1003
    • country3 1004
    • city 1005
    • org 1006
    • region 1007
    • area 1008
    • weather 1009
    • domain 1010
    • zipc 1011
    • timez 1012
    • latlng 1013
    • netspeed 1014
    • utype 1015

  Meta Tags: API ipset-ng JSON protocol