Apache NiFi

Apache NiFi is a software project from the Apache Software Foundation designed to automate the flow of data between software systems. Leveraging the concept of Extract, transform, load, it is based on the "NiagaraFiles" software previously developed by the US National Security Agency (NSA), which is also the source of a part of its present name – NiFi. It was open-sourced as a part of NSA's technology transfer program in 2014.[1][2][3][4][5]

Apache NiFi
Apache NiFi Logo
Apache NiFi Web UI
Developer(s)Apache Software Foundation,
Cloudera,
Hortonworks
Initial release2006 (2006)
Stable release
1.12.1 / 28 September 2020 (2020-09-28)
RepositoryNiFi Repository
Written inJava
Operating systemCross-platform
TypeDistributed dataflow
LicenseApache License 2.0
Websitenifi.apache.org

The software design is based on the flow-based programming model and offers features which prominently include the ability to operate within clusters, security using TLS encryption, extensibility (users can write their own software to extend its abilities) and improved usability features like a portal which can be used to view and modify behaviour visually.[6]

Software development and commercial support is currently offered by Hortonworks (now merged into Cloudera),[7] who acquired NiFi's originator, Onyara Inc.[8]

Components
NiFi - Software Components

NiFi is a Java program that runs within a Java virtual machine running on a server.[9] The prominent components of Nifi are

  • Web Server - the HTTP-based component used to visually control the software and monitor the events happening within
  • Flow Controller - serves as the brains of NiFi's behaviour. Controls the running of Nifi extensions and schedules allocation of resources for this to happen.
  • Extensions - various plugins that allow Nifi to interact with various kinds of systems
  • FlowFile repository - used by NiFi to maintain and track status of the currently active FlowFile Or the information that NiFi is helping move between systems.
  • Content repository - the data in transit is maintained here
  • Provenance repository - data relating to the provenance of the data flowing through the system is maintained here.

Integration into commercial software
  • In February 2017, HPE's SecureData for Hadoop and IoT software became Industry's first commercial product to integrate NiFi[10]

See also

References
  1. "NSA Releases First in Series of Software Products to Open Source Community". www.nsa.gov. Retrieved 2017-12-07.
  2. Bridgwater, Adrian (2015-07-21). "NSA 'NiFi' Big Data Automation Project Out In The Open". Forbes. Retrieved 2016-09-21.
  3. Vaughan-Nichols, Steven J. "NSA partners with Apache to release open-source data traffic program | ZDNet". ZDNet. Retrieved 2016-09-21.
  4. "NSA Source Code Leak: Information slurp tools to appear online". The Register. Retrieved 2016-09-21.
  5. Wolpe, Toby. "Hortonworks CTO on Apache NiFi: What is it and why does it matter to IoT? | ZDNet". ZDNet. Retrieved 2016-09-21.
  6. "Apache NiFi Documentation". nifi.apache.org. Retrieved 2017-12-07.
  7. "Welcome to a brand-new Cloudera". Hortonworks. 2019-01-03. Retrieved 2019-05-20.
  8. "Hortonworks buys Onyara, the company behind Apache NiFi, which was developed at NSA". VentureBeat. Retrieved 2016-09-21.
  9. "Apache NiFi Developer Guide". nifi.apache.org. Retrieved 2018-01-31.
  10. "RSA Conference: HPE looks to boost enterprise IoT security with Apache NiFi". TechRepublic. Retrieved 2018-01-31.

Sub-projects
  • Apache NiFi Registry: A complementary application that provides a central location for storage and management of shared resources across one or more instances of NiFi and/or MiNiFi, such as Templates.
  • MiNiFi: A complementary data collection approach that supplements the core tenets of NiFi in dataflow management, focusing on the collection of data at the source of its creation.
  • Flow Design System (FDS): An atomic reusable platform for providing a common set of UI/UX components for Apache NiFi, Apache NiFi Registry, Apache NiFi MiNiFi, and any other open source web applications to consume.
This article is issued from Wikipedia. The text is licensed under Creative Commons - Attribution - Sharealike. Additional terms may apply for the media files.