Risultati
203 risultati
-
italia
Data & Analytics Framework (DAF) - Developer Documentation
This documentation refers to the Alpha version of the DAF (released in October 2017) and it is daily updated and improved. For comments and enhancement requests about the documentation please open an issue on GitHub. The Data & Analytics Framework (DAF, in short) is an open source project developed in the context of the activities planned by the Italian Three-Year Plan for ICT in Public Administration 2017 - 2019, approved by the Italian Government in 2017. The DAF project is an attempt to establish a central Chief Data Officer (CDO) for the Government and Public Administration. Its main goal is to promote data exchange among Italian Public Administrations (PAs), to support the diffusion of open data, and to enable data-driven policies. The framework is composed by three building blocks:. A Team of Data Experts (Data Scientists and Data Engineers), able to manage and evolve the platform and to provide support to PA on their analytics and data management activities in a consultancy fashion. A Regulatory Framework, that institutionalizes this activity at government level, and gives the proper mandate to the PA that will manage the DAF, in compliance with privacy policy. This documentation is focused on the Big Data Platform, and we’ll refer to it as DAF for the sake of simplicity. The Italian instance of the DAF is developed and maintained by the DAF Team (part of the Digital Transformation Team of the Italian Government), composed by data scientists and data engineers, which uses and evolves the framework: to analyze data, to create machine learning models and to build data applications/visualization products. The DAF is designed to be easily re-usable in other countries and other application domains. It exposes the following data management and analytics functionalities:. a catalog of open-data datasets based on CKAN;. a content management system for data stories, which are a kind of blog post that integrates interactive charts (made using the DAF) with a narrative description of the analysis made;. community tools to collaborate and learn how to use the platform;. a Private Dataportal, a web application with the following features:. a catalog of all datasets the user can access;. an ingestion form to govern (insert, edit, delete) datasets information and setup ingestion procedures;. data visualization and dashboard tools;. a data science notebook;. a Hadoop Cluster with typical applications to centralize and store, manipulate and standardize and re-distribute data and insights;. a Multi-tenant architecture, based on Kerberos and LDAP. The DAF is under development. This is a snapshot of the roadmap:. By December 2017: Alpha 2 release. By January 2018: Alpha 3 release. All contributions are welcome!. Contents ...
-
italia
Installation Guide
This section provides installation information for all the components of DAF, except for the Big Data Platform. Based on your development needs, you will be able to install only the individual components you require for your task. Almost every component has dependencies on other components–this will be documented in the installation guide of each component. In general, components developed internally are available via GitHub repositories, meanwhile external ones have been dockerized with all needed dependencies and configurations. Several components are dependent on LDAP and/or FreeIPA. In this case, we offer you three alternatives: a dockerized LDAP, a remote FreeIPA test server, or a dockerized FreeIPA (working with Linux only at the moment). All of them will have test accounts already created for you. The best way to have everything installed and properly configured is to use the Virtual Machine. See the Local Installation guide to know how to configure the Virtual Machine or, if you want to run only a few components, please follow the component installation guide you find at the links below ...
-
italia
Dataportal-public
Before proceeding with the installation steps, you need to install and run the following external components:. Basic Dependencies. There are no basic dependencies needed. Features Enabling Dependencies. Connection with DataStories:. daf-dataportal-backend. Installation Steps. First of all, clone the following GitHub repository:. > $ git clone https://github.com/italia/daf-dataportal-backend. Start in Debug Mode:. > npm install > npm start. Start with mock server:. > npm run mock. Start in Production Mode:. > npm run build > npm install -g serve > serve -s build ...
-
italia
Dataportal-private
Before proceeding with the installation steps, you need to install and run the following external components:. Basic Dependencies. daf-dataportal-backend. FreeIPA. CatalogManager. SecurityManager. Features Enabling Dependencies. Superset. Metabase. JupyterHub. CKAN. Installation Steps. First of all, you need to clone the following GitHub repository:. > git clone https://github.com/italia/daf-dataportal. Start in Debug Mode:. > npm install > npm start. Start with mock server:. > npm run mock. Start in Production Mode:. > npm run build > npm install -g serve > serve -s build ...
-
italia
Dataportal
The Dataportal is the user interface of the DAF, and add additional functionalities to the big data infrastructure, such as: a dataset catalog, content management system, business intelligence tools, an interactive notebook to perform data analysis and much more. It is made of two components: a public webapp (Dataportal-public) to serve public content, and a private one (Dataportal-private) to access analytics tools and other content creation services. For installation instructions, please refer to the following sections ...
-
italia
List of microservices
The DAF Big Data platform is developed by adopting a Micro-service Architecture pattern. This is the list of the developed microservices: ...
-
italia
Frontend Manager
… ...
-
italia
Dataset Manager
[TBD] ...
-
italia
Catalog Manager
[TBD] ...
-
italia
Security Manager
[TBD] ...
-
italia
Ingestion Manager
[TBD] ...
-
italia
Storage Manager
[TBD] ...