YAML

YAML (a recursive acronym for "YAML Ain't Markup Language") is a human-readable data-serialization language. It is commonly used for configuration files and in applications where data is being stored or transmitted. YAML targets many of the same communications applications as Extensible Markup Language (XML) but has a minimal syntax which intentionally differs from SGML.[1] It uses both Python-style indentation to indicate nesting, and a more compact format that uses [...] for lists and {...} for maps[1] making YAML 1.2 a superset of JSON.[2]

For the CSS framework, see YAML (framework).
YAML
Filename extensions.yaml, .yml
Internet media typeNot registered
Initial release11 May 2001 (2001-05-11)
Latest release
1.2 (Third Edition)
(1 October 2009 (2009-10-01))
Type of formatData interchange
Open format?Yes
Websiteyaml.org

Custom data types are allowed, but YAML natively encodes scalars (such as strings, integers, and floats), lists, and associative arrays (also known as maps, dictionaries or hashes). These data types are based on the Perl programming language, though all commonly used high-level programming languages share very similar concepts. The colon-centered syntax, used for expressing key-value pairs, is inspired by electronic mail headers as defined in RFC 822, and the document separator --- is borrowed from MIME (RFC 2046). Escape sequences are reused from C, and whitespace wrapping for multi-line strings is inspired by HTML. Lists and hashes can contain nested lists and hashes, forming a tree structure; arbitrary graphs can be represented using YAML aliases (similar to XML in SOAP).[1] YAML is intended to be read and written in streams, a feature inspired by SAX.[1]

Support for reading and writing YAML is available for many programming languages.[3] Some source-code editors such as Emacs[4] and various integrated development environments[5][6][7] have features that make editing YAML easier, such as folding up nested structures or automatically highlighting syntax errors.

The official recommended filename extension for YAML files has been .yaml[8] since 2006.[9]

History and name

YAML (/ˈjæməl/, rhymes with camel[2]) was first proposed by Clark Evans in 2001,[10] who designed it together with Ingy döt Net[11] and Oren Ben-Kiki.[11] Originally YAML was said to mean Yet Another Markup Language,[12] referencing its purpose as a markup language with the yet another construct, but it was then repurposed as YAML Ain't Markup Language, a recursive acronym, to distinguish its purpose as data-oriented, rather than document markup.

Design

Syntax

A cheat sheet and full specification are available at the official site.[13] The following is a synopsis of the basic elements.

YAML accepts the entire Unicode character set, except for some control characters, and may be encoded in UTF-8, UTF-16 and UTF-32. (Though UTF-32 is not mandatory, it is required for a parser to have JSON compatibility.)[14]

  • Whitespace indentation is used for denoting structure; however, tab characters are not allowed as part of that indentation.
  • Comments begin with the number sign (#), can start anywhere on a line and continue until the end of the line. Comments must be separated from other tokens by whitespace characters.[15] If # characters appear inside of a string, then they are number sign (#) literals.
  • List members are denoted by a leading hyphen (-) with one member per line.
    • A list can also be specified by enclosing text in square brackets ([...]) with each entry separated by a comma.
  • An associative array entry is represented using colon space in the form key: value with one entry per line. YAML requires the colon be followed by a space so that scalar values such as http://www.wikipedia.org can generally be represented without needing to be enclosed in quotes.
    • A question mark can be used in front of a key, in the form "?key: value" to allow the key to contain leading dashes, square brackets, etc., without quotes.
    • An associative array can also be specified by text enclosed in curly braces ({...}), with keys separated from values by colon and the entries separated by commas (spaces are not required to retain compatibility with JSON).
  • Strings (one type of scalar in YAML) are ordinarily unquoted, but may be enclosed in double-quotes ("), or single-quotes (').
    • Within double-quotes, special characters may be represented with C-style escape sequences starting with a backslash (\). According to the documentation the only octal escape supported is \0.
  • Block scalars are delimited with indentation with optional modifiers to preserve (|) or fold (>) newlines.
  • Multiple documents within a single stream are separated by three hyphens (---).
    • Three periods (...) optionally end a document within a stream.
  • Repeated nodes are initially denoted by an ampersand (&) and thereafter referenced with an asterisk (*).
  • Nodes may be labeled with a type or tag using a double exclamation mark (!!) followed by a string, which can be expanded into a URI.
  • YAML documents in a stream may be preceded by 'directives' composed of a percent sign (%) followed by a name and space-delimited parameters. Two directives are defined in YAML 1.1:
    • The %YAML directive is used for identifying the version of YAML in a given document.
    • The %TAG directive is used as a shortcut for URI prefixes. These shortcuts may then be used in node type tags.

Two additional sigil characters are reserved in YAML for possible future specification: the at sign (@) and backtick (`).

Basic components

YAML offers an "in-line" style for denoting associative arrays and lists. Here is a sample of the components.

Conventional block format uses a hyphen+space to begin a new item in list.

--- # Favorite movies
- Casablanca
- North by Northwest
- The Man Who Wasn't There

Optional inline format is delimited by comma+space and enclosed in brackets (similar to JSON).[16]

--- # Shopping list
[milk, pumpkin pie, eggs, juice]

Keys are separated from values by a colon+space. Indented blocks, common in YAML data files, use indentation and new lines to separate the key/value pairs. Inline Blocks, common in YAML data streams, use comma+space to separate the key/value pairs between braces.

--- # Indented Block
  name: John Smith
  age: 33
--- # Inline Block
{name: John Smith, age: 33}

Strings do not require quotation marks. There are two ways to write multi-line strings, one preserving newlines (using the | character) and one that folds the newlines (using the > character), both followed by a newline character.

data: |
   There once was a tall man from Ealing
   Who got on a bus to Darjeeling
       It said on the door
       "Please don't sit on the floor"
   So he carefully sat on the ceiling

By default, the leading indentation (of the first line) and trailing whitespace is stripped, though other behavior can be explicitly specified.

data: >
   Wrapped text
   will be folded
   into a single
   paragraph

   Blank lines denote
   paragraph breaks

Folded text converts newlines to spaces and removes leading whitespace.

--- # The Smiths
- {name: John Smith, age: 33}
- name: Mary Smith
  age: 27
- [name, age]: [Rae Smith, 4]   # sequences as keys are supported
--- # People, by gender
men: [John Smith, Bill Jones]
women:
  - Mary Smith
  - Susan Williams

Objects and lists are important components in yaml and can be mixed. The first example is a list of key-value objects, all people from the Smith family. The second lists them by gender; it is a key-value object containing two lists.

Advanced components

Two features that distinguish YAML from the capabilities of other data-serialization languages are structures[17] and data typing.

YAML structures enable storage of multiple documents within a single file, usage of references for repeated nodes, and usage of arbitrary nodes as keys.[17]

For clarity, compactness, and avoiding data entry errors, YAML provides node anchors (using &) and references (using *). References to the anchor work for all data types (see the ship-to reference in the example below).

Below is an example of a queue in an instrument sequencer in which two steps are reused repeatedly without being fully described each time.

--- # Sequencer protocols for Laser eye surgery
- step:  &id001                  # defines anchor label &id001
    instrument:      Lasik 2000
    pulseEnergy:     5.4
    pulseDuration:   12
    repetition:      1000
    spotSize:        1mm

- step: &id002
    instrument:      Lasik 2000
    pulseEnergy:     5.0
    pulseDuration:   10
    repetition:      500
    spotSize:        2mm
- step: *id001                   # refers to the first step (with anchor &id001)
- step: *id002                   # refers to the second step
- step: *id002

Explicit data typing is seldom seen in the majority of YAML documents since YAML autodetects simple types. Data types can be divided into three categories: core, defined, and user-defined. Core are ones expected to exist in any parser (e.g. floats, ints, strings, lists, maps, ...). Many more advanced data types, such as binary data, are defined in the YAML specification but not supported in all implementations. Finally YAML defines a way to extend the data type definitions locally to accommodate user-defined classes, structures or primitives (e.g. quad-precision floats).

YAML autodetects the datatype of the entity, but sometimes one wants to cast the datatype explicitly. The most common situation is where a single-word string that looks like a number, boolean or tag requires disambiguation by surrounding it with quotes or using an explicit datatype tag.

---
a: 123                     # an integer
b: "123"                   # a string, disambiguated by quotes
c: 123.0                   # a float
d: !!float 123             # also a float via explicit data type prefixed by (!!)
e: !!str 123               # a string, disambiguated by explicit type
f: !!str Yes               # a string via explicit type
g: Yes                     # a boolean True (yaml1.1), string "Yes" (yaml1.2)
h: Yes we have No bananas  # a string, "Yes" and "No" disambiguated by context.

Not every implementation of YAML has every specification-defined data type. These built-in types use a double-exclamation sigil prefix (!!). Particularly interesting ones not shown here are sets, ordered maps, timestamps, and hexadecimal. Here's an example of base64-encoded binary data.

---
picture: !!binary |
  R0lGODdhDQAIAIAAAAAAANn
  Z2SwAAAAADQAIAAACF4SDGQ
  ar3xxbJ9p0qa7R0YxwzaFME
  1IAADs=

Many implementations of YAML can support user-defined data types for object serialization. Local data types are not universal data types but are defined in the application using the YAML parser library. Local data types use a single exclamation mark (!).

---
myObject: !myClass { name: Joe, age: 15 }

Example

Data-structure hierarchy is maintained by outline indentation.

---
receipt:     Oz-Ware Purchase Invoice
date:        2012-08-06
customer:
    first_name:   Dorothy
    family_name:  Gale

items:
    - part_no:   A4786
      descrip:   Water Bucket (Filled)
      price:     1.47
      quantity:  4

    - part_no:   E1628
      descrip:   High Heeled "Ruby" Slippers
      size:      8
      price:     133.7
      quantity:  1

bill-to:  &id001
    street: |
            123 Tornado Alley
            Suite 16
    city:   East Centerville
    state:  KS

ship-to:  *id001

specialDelivery:  >
    Follow the Yellow Brick
    Road to the Emerald City.
    Pay no attention to the
    man behind the curtain.
...

Notice that strings do not require enclosure in quotation marks. The specific number of spaces in the indentation is unimportant as long as parallel elements have the same left justification and the hierarchically nested elements are indented further. This sample document defines an associative array with 7 top level keys: one of the keys, "items", contains a 2-element list, each element of which is itself an associative array with differing keys. Relational data and redundancy removal are displayed: the "ship-to" associative array content is copied from the "bill-to" associative array's content as indicated by the anchor (&) and reference (*) labels. Optional blank lines can be added for readability. Multiple documents can exist in a single file/stream and are separated by ---. An optional ... can be used at the end of a file (useful for signaling an end in streamed communications without closing the pipe).

Features

Indented delimiting

Because YAML primarily relies on outline indentation for structure, it is especially resistant to delimiter collision. YAML's insensitivity to quotation marks and braces in scalar values means one may embed XML, JSON or even YAML documents inside a YAML document by simply indenting it in a block literal (using | or >):

---
example: >
        HTML goes into YAML without modification
message: |

        <blockquote style="font: italic 1em serif">
        <p>"Three is always greater than two,
           even for large values of two"</p>
        <p>--Author Unknown</p>
        </blockquote>
date: 2007-06-01

YAML may be placed in JSON by quoting and escaping all interior quotation marks. YAML may be placed in XML by escaping reserved characters (<, >, &, ', ") and converting whitespace, or by placing it in a CDATA section.

Non-hierarchical data models

Unlike JSON, which can only represent data in a hierarchical model with each child node having a single parent, YAML also offers a simple relational scheme that allows repeats of identical data to be referenced from two or more points in the tree rather than entered redundantly at those points. This is similar to the facility IDREF built into XML.[18] The YAML parser then expands these references into the fully populated data structures they imply when read in, so whatever program is using the parser does not have to be aware of a relational encoding model, unlike XML processors, which do not expand references. This expansion can enhance readability while reducing data entry errors in configuration files or processing protocols where many parameters remain the same in a sequential series of records while only a few vary. An example being that "ship-to" and "bill-to" records in an invoice are nearly always the same data.

Practical considerations

YAML is line-oriented and thus it is often simple to convert the unstructured output of existing programs into YAML format while having them retain much of the look of the original document. Because there are no closing tags, braces, or quotation marks to balance, it is generally easy to generate well-formed YAML directly from distributed print statements within unsophisticated programs. Likewise, the whitespace delimiters facilitate quick-and-dirty filtering of YAML files using the line-oriented commands in grep, AWK, Perl, Ruby, and Python.

In particular, unlike markup languages, chunks of consecutive YAML lines tend to be well-formed YAML documents themselves. This makes it very easy to write parsers that do not have to process a document in its entirety (e.g. balancing opening and closing tags and navigating quoted and escaped characters) before they begin extracting specific records within. This property is particularly expedient when iterating in a single, stateless pass, over records in a file whose entire data structure is too large to hold in memory, or for which reconstituting the entire structure to extract one item would be prohibitively expensive.

Counterintuitively, although its indented delimiting might seem to complicate deeply nested hierarchies, YAML handles indents as small as a single space, and this may achieve better compression than markup languages. Additionally, extremely deep indentation can be avoided entirely by either: 1) reverting to "inline style" (i.e. JSON-like format) without the indentation; or 2) using relational anchors to unwind the hierarchy to a flat form that the YAML parser will transparently reconstitute into the full data structure.

Security

YAML is purely a data-representation language and thus has no executable commands.[19] While validation and safe parsing is inherently possible in any data language, implementation is such a notorious pitfall that YAML's lack of an associated command language may be a relative security benefit.

However, YAML allows language-specific tags so that arbitrary local objects can be created by a parser that supports those tags. Any YAML parser that allows sophisticated object instantiation to be executed opens the potential for an injection attack. Perl parsers that allow loading of objects of arbitrary classes create so-called "blessed" values. Using these values may trigger unexpected behavior, e.g. if the class uses overloaded operators. This may lead to execution of arbitrary Perl code.

The situation is similar for Python or Ruby parsers. According to the PyYAML documentation:[20]

Note that the ability to construct an arbitrary Python object may be dangerous if you receive a YAML document from an untrusted source such as the Internet. The function yaml.safe_load limits this ability to simple Python objects like integers or lists. [...]

PyYAML allows you to construct a Python object of any type. Even instances of Python classes can be constructed using the !!python/object tag.

Data processing and representation

The YAML specification identifies an instance document as a "Presentation" or "character stream".[21] The primary logical structures in a YAML instance document are scalars, sequences, and mappings.[22] The YAML specification also indicates some basic constraints that apply to these primary logical structures. For example, according to the specification, mapping keys do not have an order. In every case where node order is significant, a sequence must be used.[23]

Moreover, in defining conformance for YAML processors, the YAML specification defines two primary operations: dump and load. All YAML-compliant processors must provide at least one of these operations, and may optionally provide both.[24] Finally, the YAML specification defines an information model or "representation graph", which must be created during processing for both dump and load operations, although this representation need not be made available to the user through an API.[25]

Comparison with other serialization formats

Comparison with JSON

JSON syntax is a basis of YAML version 1.2, which was promulgated with the express purpose of bringing YAML "into compliance with JSON as an official subset".[2] Though prior versions of YAML were not strictly compatible,[26] the discrepancies were rarely noticeable, and most JSON documents can be parsed by some YAML parsers such as Syck.[27] This is because JSON's semantic structure is equivalent to the optional "inline-style" of writing YAML. While extended hierarchies can be written in inline-style like JSON, this is not a recommended YAML style except when it aids clarity.

YAML has many additional features lacking in JSON, including comments, extensible data types, relational anchors, strings without quotation marks, and mapping types preserving key order.

Comparison with TOML

On the one hand, YAML is much more complex compared to TOML - the YAML specification was pointed out to have 23,449 words, while the TOML specification had only 3,339 words.[28] On the other hand, YAML is less verbose, more DRY, syntactically less noisy, and the hierarchy of a document is obvious from the indentation[29].

Comparison with XML

YAML lacks the notion of tag attributes that are found in XML. Instead YAML has extensible type declarations (including class types for objects).

YAML itself does not have XML's language-defined document schema descriptors that allow, for example, a document to self-validate. However, there are several externally defined schema descriptor languages for YAML (e.g. Doctrine, Kwalify and Rx) that fulfill that role. Moreover, the semantics provided by YAML's language-defined type declarations in the YAML document itself frequently relaxes the need for a validator in simple, common situations. Additionally, YAXML, which represents YAML data structures in XML, allows XML schema importers and output mechanisms like XSLT to be applied to YAML.

Software (emitters and parsers)

For fixed data structures, YAML files can simply be generated using print commands that write both the data and the YAML specific decoration. To dump varying, or complex, hierarchical data, however, a dedicated YAML emitter is preferable. Similarly, simple YAML files (e.g. key-value pairs) are readily parsed with regular expressions. For more complex, or varying, data structures, a formal YAML parser is recommended.

YAML emitters and parsers exist for many popular languages. Most of them are written in the native language itself. Some are language bindings of the C library libyaml; they may run faster. There used to be another C library, called Syck, written and orphaned by why the lucky stiff: it is unmaintained, there is no authoritative source bundle, and the web site has been hijacked. Hence the only recommendable C library is libyaml. It was originally developed by Kirill Simonov. In 2018, development was resumed by the new maintainers Ian Cordasco and Ingy döt Net.[30]

C++ programmers have the choice between the C library libyaml and the C++ library libyaml-cpp. Both have completely independent code bases and completely different APIs. The library libyaml-cpp still has a major version number of 0, indicating that the API may change at any moment, as happened indeed after version 0.3. There is a grammar-focused implementation written in C#, with an aim on extensions for the nested elements.[31]

Some implementations of YAML, such as Perl's YAML.pm, will load an entire file (stream) and parse it en masse. Other implementations like PyYaml are lazy and iterate over the next document only upon request. For very large files in which one plans to handle the documents independently, instantiating the entire file before processing may be prohibitive. Thus in YAML.pm, occasionally one must chunk a file into documents and parse those individually. YAML makes this easy, since this simply requires splitting on the document end marker, which is defined as three periods on a line by themselves. This marker is forbidden in content.[32]

Criticism

YAML has been criticized for its significant whitespace, confusing features, insecure defaults, and its complex and ambiguous specification:[28][33][34]

  • Configuration files can execute commands or load contents without the users realizing it.[28]
  • Editing large YAML files is difficult, as indentation errors can go unnoticed.[28]
  • Type autodetection is a source of errors. For example, unquoted Yes and NO are converted to booleans; software version numbers might be converted to floats.[28][35]
  • Truncated files are often interpreted as valid YAML due to the absence of terminators.
  • The complexity of the standard led to inconsistent implementations and making the language non-portable.[28][36]

The perceived flaws and complexity of YAML has led to the appearance of StrictYAML, which uses a restricted subset of the YAML specification.[35]

See also

References
  1. "Yet Another Markup Language (YAML) 1.0 / Working Draft". 10 Dec 2001.
  2. "YAML Ain't Markup Language (YAML) Version 1.2". YAML.org. Retrieved 2019-05-29.
  3. "The Official YAML Web Site". yaml.org.
  4. "Yaml Mode". EmacsWiki. 2015-06-12. Retrieved 2016-12-05.
  5. aukaost. "Pretty YAML - Packages - Package Control". Packagecontrol.io. Retrieved 2016-12-05.
  6. "yaml | Eclipse Plugins, Bundles and Products - Eclipse Marketplace". Marketplace.eclipse.org. Retrieved 2016-12-05.
  7. Ruth Kusterer. "NetBeans IDE - Ruby and Ruby on Rails Development". Netbeans.org. Retrieved 2016-12-05.
  8. "YAML Ain't Markup Language". yaml.org.
  9. "YAML Ain't Markup Language". web.archive.org. September 24, 2006.
  10. Evans, Clark (May 11, 2001). "YAML Draft 0.1". Yahoo! Tech groups: sml-dev. Retrieved 2019-03-21.
  11. "YAML Ain't Markup Language: About". YAML.org. Retrieved 2019-05-29.
  12. "Yet Another Markup Language (YAML) 1.0". YAML.org. Retrieved 2019-05-29.
  13. "YAML 1.1 Reference Card". YAML.org. Retrieved 2019-05-29.
  14. "YAML Ain't Markup Language (YAML) Version 1.2". YAML.org. Retrieved 2019-05-29.
  15. "YAML Ain't Markup Language (YAML) Version 1.2". YAML.org. Retrieved 2019-05-29.
  16. "Cloud Based Management apps". JigoCloud.com. Archived from the original on 2016-09-17. Retrieved 2016-09-28.
  17. "YAML 1.2 specification of Structures". YAML.org. Retrieved 2019-05-29.
  18. "Extensible Markup Language (XML) 1.0 (Second Edition)". W3.org. Retrieved 27 May 2015.
  19. A proposed "yield" tag would provide for simple arithmetic calculations.
  20. "PyYAML Documentation, Loading YAML". Pyyaml.org. Retrieved 2016-09-28.
  21. "Ain't Markup Language (YAML) Version 1.1". YAML.org. Retrieved 2019-05-29.
  22. Additional, optional-use, logical structures are enumerated in the YAML types repository."Language-Independent Types for YAML Version 1.1". YAML.org. Retrieved 2019-05-29. The tagged types in the YAML types repository are optional and therefore not essential for conformant YAML processors. "The use of these tags is not mandatory."
  23. "YAML Ain't Markup Language (YAML) Version 1.1". YAML.org. Retrieved 2019-05-29.
  24. "Ain't Markup Language (YAML) Version 1.1". YAML.org. Retrieved 2019-05-29.
  25. "YAML Ain't Markup Language (YAML) Version 1.1". YAML.org. Retrieved 2019-05-29.
  26. The incompatibilities were as follows: JSON allows extended character sets like UTF-32 and had incompatible unicode character escape syntax relative to YAML; YAML required a space after separators like comma, equals, and colon while JSON does not. Some non-standard implementations of JSON extend the grammar to include Javascript's /*...*/ comments. Handling such edge cases may require light pre-processing of the JSON before parsing as in-line YAML. See also .
  27. Parsing JSON with SYCK. Note that e.g. Symfony's YAML parser does not support line breaks inside [] or {} structures, which is a major incompatibility with JSON.
  28. Tournoij, Martin (4 Sep 2016). "YAML: probably not so great after all". Retrieved 16 May 2019.
  29. What is wrong with TOML?
  30. [email protected], mail of June 27, 2018.
  31. "YAML Grammar for Lexepars".
  32. "YAML Ain't Markup Language (YAML) Version 1.2 # 9.1.2 Document Markers". YAML.org. Retrieved 2019-05-29.
  33. "That's a lot of YAML". Retrieved 16 May 2019.
  34. "YAML sucks". Retrieved 16 May 2019.
  35. "The Norway Problem - why StrictYAML refuses to do implicit typing and so should you". Retrieved 3 June 2020.
  36. "YAML Test Matrix".
This article is issued from Wikipedia. The text is licensed under Creative Commons - Attribution - Sharealike. Additional terms may apply for the media files.