Main data and configuration file formats

When you need to pass configuration data to an application, there are many file formats available, some more well-known than others.

There is no perfect format, the goal is always to have a format that is both complete, very readable and easy to pick up, but the specifics of each ecosystem mean that each of them is relevant.

The goal of this article is to introduce you to and familiarize you with formats you may not necessarily know, which is why the same dataset will be used as an example each time.

INI Format

This is a format made mostly of key-value pairs but with the possibility of separating them into sections. It is mainly used in the Microsoft environment (under the .ini extension), but a large part of the configuration files on Linux are represented by a simplified key-value format.

;List of all endpoints

[request]
id=request
protocol=https 
domain=example.com
uri=/request, 
description="Request enpoint"

[search]
id=search
protocol=https 
domain=google.com
uri=/search, 
description="Search enpoint"

The INI format is unusual because, compared to the other formats in this list, it is the only one to be "informal" as it is not defined by a strict specification.

CSV Format

The CSV format is a format originally designed to represent two-dimensional tables, widely used to export data present in spreadsheets (such as Excel).

This format is very simple and its explicit name (comma-separated values) describes very well how it works. Comments are not meant to appear in a CSV file, only values, and commas!

id, protocol, domain, uri, description
request, https, example.com, /request, Request endpoint
search, https, google.com, /search, Search enpoint

XML Format

This format is very present in the Java ecosystem, but also on the web (less and less). While XML is verbose, it has the advantage of being very explicit, and above all it is the only format that provides in its specification a validation technology for the file's structure with DTD!

<?xml version="1.0" encoding="UTF-8" ?>
<!-- List of all endpoints -->
<endpoints>
	<endpoint id="request">
    	<protocol>https</protocol>
        <domain>example.com</domain>
        <uri>/request</uri>
        <description>Request endpoint</description>
    </endpoint>
    <endpoint id="search">
    	<protocol>https</protocol>
        <domain>google.com</domain>
        <uri>/search</uri>
        <description>Search endpoint</description>
    </endpoint>
</endpoints>

JSON Format

If you've done Javascript, then you know what the JSON format is. Actually called "JavaScript Object Notation" it is used throughout the entire Javascript ecosystem, NodeJS and is taking up more and more space on the web in general.

[{
    "id": "request",
    "protocol": "https", 
    "domain":"example.com"
    "uri":"/request", 
    "description":"Request enpoint"
},
{
    "id":"search",
    "protocol":"https" 
    "domain":"google.com"
    "uri":"/search", 
    "description":"Search enpoint"
}]

The JSON format is also used by some NoSQL databases to store data and build queries, as is the case with MongoDB for example.

YAML Format

YAML (whose name is a recursive acronym for "YAML Ain't Markup Language") is a format that pits ease of reading against a certain complexity of writing due to the many data types, the mandatory space indentation, etc...

The data types and structures added to YAML 1.2 notably make it a "superset" of JSON with the arrival of lists "[...]" and maps "{...}", so it is possible to describe complex data architectures.

---# List of all endpoints

- request:
    protocol: https
    domain: example.com
    uri: /request 
    description: Request enpoint 
- search:
    protocol: https
    domain: google.com
    uri: /search
    description: Search enpoint
    

TOML Format

This format, an acronym for "Tom's Obvious, Minimal Language" is a newcomer (it has existed since 2013, but version 1.0 dates from January 2021) and combines the good sides of several of the formats mentioned above.

For now, the uses of this format remain rather few, but it is used, for example, for configuring the Cargo tool, Rust's package manager.

#List of all endpoints

[endpoints]
    [endpoint.request]
        protocol="https" 
        domain="example.com"
        uri="/request", 
        description="Request enpoint"

    [endpoint.search]
        protocol=https 
        domain=google.com
        uri=/search, 
        description="Search enpoint"

TOML offers a complex but readable and easily editable structure, has comments, indentation but optional, and remains much easier to implement than YAML given its lighter specification.