cgiar platform for big data in agriculture · data in agriculture: building a global agricultural...

50
CGIAR PLATFORM FOR BIG DATA IN AGRICULTURE: BUILDING A GLOBAL AGRICULTURAL DATA AND OPEN INNOVATION ECOSYSTEM FOR IMPACT

Upload: others

Post on 31-Dec-2020

5 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: CGIAR PLATFORM FOR BIG DATA IN AGRICULTURE · data in agriculture: building a global agricultural data and open innovation ... power of big data analytics through inspiring and innovative

CGIAR PLATFORM FOR BIG

DATA IN AGRICULTURE:

BUILDING A GLOBAL

AGRICULTURAL DATA AND

OPEN INNOVATION

ECOSYSTEM FOR IMPACT

Page 2: CGIAR PLATFORM FOR BIG DATA IN AGRICULTURE · data in agriculture: building a global agricultural data and open innovation ... power of big data analytics through inspiring and innovative

CGIAR Big Data Platform – An overview

Brian King – CIAT

Page 3: CGIAR PLATFORM FOR BIG DATA IN AGRICULTURE · data in agriculture: building a global agricultural data and open innovation ... power of big data analytics through inspiring and innovative

MODULES

rganiORGANIZESUPPORT AND

IMPROVE DATA

GENERATION, ACCESS,

AND MANAGEMENT

ConveneCONVENE

COLLABORATE AND

CONVENE AROUND BIG

DATA AND

AGRICULTURAL

DEVELOPMENT

InspireINSPIRE

DEMONSTRATING THE

POWER OF BIG DATA

ANALYTICS THROUGH

INSPIRING AND

INNOVATIVE PROJECTS

Page 4: CGIAR PLATFORM FOR BIG DATA IN AGRICULTURE · data in agriculture: building a global agricultural data and open innovation ... power of big data analytics through inspiring and innovative

170,000 publications & 27,000 datasets discoverable

GARDIAN also includes guidance, workflows and tools to make CGIAR data

assets open and Findable, Accessible, Interoperable, Reusable (FAIR).

ORGANIZE

SUPPORT AND IMPROVE DATA GENERATION, ACCESS, AND MANAGEMENT IN CGIAR

DIGITAL TRANSFORMATION | THROUGH DATA STANDARDS AND SHARING

Collaborative GARDIAN Labs (CG Labs) offers researchers a secure

analytic environment to find data and collaborate on

analyses within the GARDIAN ecosystem, integrating single sign-on and

Globus, a service enabling secure data sharing.

CGIAR EXPERT FINDER

Enables discovery of CGIAR research expertise organized by people, geographies, Centers, funding agencies, and

publications.

Page 5: CGIAR PLATFORM FOR BIG DATA IN AGRICULTURE · data in agriculture: building a global agricultural data and open innovation ... power of big data analytics through inspiring and innovative

ANNUAL CONVENTION

1,800 attendees +

8,000 online viewers, to date

Approx. 65% external to CGIAR,

catalysing alliances with stakeholders

across the technology and agricultural

sectors

CONVENE

COLLABORATE AND CONVENE AROUND BIG DATA AND AGRICULTURAL DEVELOPMENT

DIGITAL TRANSFORMATION | THROUGH ALLIANCES AND COMMUNITIES OF PRACTICE

SHARED SERVICES

Empowering CGIAR and its

community to deliver on the potential of

big data to bring results for smallholder

agriculture.

COMMUNITIES OF PRACTICE

3.5K experts across six communities

with approx. 80% external to CGIAR

Page 6: CGIAR PLATFORM FOR BIG DATA IN AGRICULTURE · data in agriculture: building a global agricultural data and open innovation ... power of big data analytics through inspiring and innovative

THE INSPIRE CHALLENGE

Revealing

Food

Systems

Sensing & Renewing

EcosystemsSustaining Farm

Income

Measuring &

Building Resilience

Page 7: CGIAR PLATFORM FOR BIG DATA IN AGRICULTURE · data in agriculture: building a global agricultural data and open innovation ... power of big data analytics through inspiring and innovative

DIGITAL TRANSFORMATION THROUGH DIGITAL INNOVATION

2.6M USD in total grants

awarded

600,000 USDraised --and growing-- from

external investors

11 CGIAR Centers & CRPs

partnered with

31 expert industry partners

500 Inspire Challenge

applications

18 grants awarded to

innovative projects

3 Rapid Response grants

awarded to mitigate

COVID-19 impacts

INSPIREDEMONSTRATING THE POWER OF BIG DATA ANALYTICS THROUGH INSPIRING AND INNOVATIVE PROJECTS

Page 8: CGIAR PLATFORM FOR BIG DATA IN AGRICULTURE · data in agriculture: building a global agricultural data and open innovation ... power of big data analytics through inspiring and innovative

Show me the data!

Medha Devare – IFPRI

Jawoo Koo - IFPRI

Page 9: CGIAR PLATFORM FOR BIG DATA IN AGRICULTURE · data in agriculture: building a global agricultural data and open innovation ... power of big data analytics through inspiring and innovative

FUTURE FARMS…

BIG DATA-DRIVEN SOLUTIONS.

Find data Visualize

Interpret Analyze

Aggregate

Page 10: CGIAR PLATFORM FOR BIG DATA IN AGRICULTURE · data in agriculture: building a global agricultural data and open innovation ... power of big data analytics through inspiring and innovative

https://www.openaire.eu/how-to-make-your-data-fair

RESEARCH DATA | OPEN BY DEFAULT

Page 11: CGIAR PLATFORM FOR BIG DATA IN AGRICULTURE · data in agriculture: building a global agricultural data and open innovation ... power of big data analytics through inspiring and innovative

….

DIGITAL TOOLS TO

COLLECT STANDARDS-

COMPLIANT DATA

WORKFLOWS TO

COMPLY WITH

STANDARDS

TOOLS FOR

DATA ANALYSIS,

VISUALIZATION

WIDE DATA

DISCOVERY, USE

DATA

DISCOVERY

PORTAL

Page 12: CGIAR PLATFORM FOR BIG DATA IN AGRICULTURE · data in agriculture: building a global agricultural data and open innovation ... power of big data analytics through inspiring and innovative

nutrition women

Page 13: CGIAR PLATFORM FOR BIG DATA IN AGRICULTURE · data in agriculture: building a global agricultural data and open innovation ... power of big data analytics through inspiring and innovative

FCDO

USAID - DDL

data.gov.in

World Bank

IFPRI

ILRI

CIP

Page 14: CGIAR PLATFORM FOR BIG DATA IN AGRICULTURE · data in agriculture: building a global agricultural data and open innovation ... power of big data analytics through inspiring and innovative
Page 15: CGIAR PLATFORM FOR BIG DATA IN AGRICULTURE · data in agriculture: building a global agricultural data and open innovation ... power of big data analytics through inspiring and innovative
Page 16: CGIAR PLATFORM FOR BIG DATA IN AGRICULTURE · data in agriculture: building a global agricultural data and open innovation ... power of big data analytics through inspiring and innovative

AgroFIMS

Agronomy Field Information

Management System

Generate standardized field books

to collect agronomic data that is…

https://apps.cipotato.org/hidapagrofims

Page 17: CGIAR PLATFORM FOR BIG DATA IN AGRICULTURE · data in agriculture: building a global agricultural data and open innovation ... power of big data analytics through inspiring and innovative

“I want to be able to explore data visually!”

Page 18: CGIAR PLATFORM FOR BIG DATA IN AGRICULTURE · data in agriculture: building a global agricultural data and open innovation ... power of big data analytics through inspiring and innovative
Page 19: CGIAR PLATFORM FOR BIG DATA IN AGRICULTURE · data in agriculture: building a global agricultural data and open innovation ... power of big data analytics through inspiring and innovative
Page 20: CGIAR PLATFORM FOR BIG DATA IN AGRICULTURE · data in agriculture: building a global agricultural data and open innovation ... power of big data analytics through inspiring and innovative

“I want to be able to work with my team to find, securely share, and analyze data.”

Users must have

Globus accounts:

https://www.globus.org/

via gmail or

institutional account

https://gardian.bigdata.cgiar.org/labs.php

Page 21: CGIAR PLATFORM FOR BIG DATA IN AGRICULTURE · data in agriculture: building a global agricultural data and open innovation ... power of big data analytics through inspiring and innovative

Oganization-wide secure data sharing

and discovery services

Multi-faceted collaboration spaces at

customizable levels (project teams, groups,

labs, etc.)

Role-based access and Single Sign-On

Data sharing & discovery

CollaborationSecurity

Management

Index and search internal data to enable secured

web-based and API access to organizational assets

Private Data

COLLABORATIVE GARDIAN LABS | AN ANALYTICS PIPELINE

… and Open Source

Page 22: CGIAR PLATFORM FOR BIG DATA IN AGRICULTURE · data in agriculture: building a global agricultural data and open innovation ... power of big data analytics through inspiring and innovative

Note: Some CC0 and

CCBY datasets are

locked down and

cannot be

downloaded yet

Page 23: CGIAR PLATFORM FOR BIG DATA IN AGRICULTURE · data in agriculture: building a global agricultural data and open innovation ... power of big data analytics through inspiring and innovative

CG LABS | DOWNSCALED CMIP6 DATASET (7 TB)

Page 24: CGIAR PLATFORM FOR BIG DATA IN AGRICULTURE · data in agriculture: building a global agricultural data and open innovation ... power of big data analytics through inspiring and innovative

Click 1: Select CMIP6 Model

Click 2: Pin region

Click 3: Select time period

Click 4: Download data as GeoJSON or CSV

Page 25: CGIAR PLATFORM FOR BIG DATA IN AGRICULTURE · data in agriculture: building a global agricultural data and open innovation ... power of big data analytics through inspiring and innovative
Page 26: CGIAR PLATFORM FOR BIG DATA IN AGRICULTURE · data in agriculture: building a global agricultural data and open innovation ... power of big data analytics through inspiring and innovative

C. Porter et al., Univ. of Florida, 2019C. Porter et al., Univ. of Florida, 2019

The data variables map to the

ICASA Data Dictionary, and

Agronomy Ontology and other

ontologies. AgMIP translation

tools can therefore translate

the data for several widely-

used crop simulation models.

Ava’s GARDIAN search reveals

37 field experiments with

several crops.

COMING SOON | TO CG LABS…

“I need to recommend sustainable intensification practices for West Africa.”

Page 27: CGIAR PLATFORM FOR BIG DATA IN AGRICULTURE · data in agriculture: building a global agricultural data and open innovation ... power of big data analytics through inspiring and innovative

Data shmata…

Page 28: CGIAR PLATFORM FOR BIG DATA IN AGRICULTURE · data in agriculture: building a global agricultural data and open innovation ... power of big data analytics through inspiring and innovative

Profitability of fertilizer use in Africa

A case-study in data re-use

Camila Bonilla Cedrez – CIAT

Jordan Chamberlin – CIMMYT

Robert Hijmans – UC Davis

Page 29: CGIAR PLATFORM FOR BIG DATA IN AGRICULTURE · data in agriculture: building a global agricultural data and open innovation ... power of big data analytics through inspiring and innovative

Fertilizer use in Africa

Why is fertilizer use low ?

low crop response (poor soil / climate)?

unfavorable prices?

uncertainty (that can perhaps be insured)?

We need site-specific

crop response to fertilizer model

and input and output price data

Page 30: CGIAR PLATFORM FOR BIG DATA IN AGRICULTURE · data in agriculture: building a global agricultural data and open innovation ... power of big data analytics through inspiring and innovative

108 data sets

760 locations

12,000 observations

Findable and Accessible data

Page 31: CGIAR PLATFORM FOR BIG DATA IN AGRICULTURE · data in agriculture: building a global agricultural data and open innovation ... power of big data analytics through inspiring and innovative

Interoperable?

Page 32: CGIAR PLATFORM FOR BIG DATA IN AGRICULTURE · data in agriculture: building a global agricultural data and open innovation ... power of big data analytics through inspiring and innovative

Aggregated (raw) data

Page 33: CGIAR PLATFORM FOR BIG DATA IN AGRICULTURE · data in agriculture: building a global agricultural data and open innovation ... power of big data analytics through inspiring and innovative

N response

Reuse: Machine learning predictions

P response

Page 34: CGIAR PLATFORM FOR BIG DATA IN AGRICULTURE · data in agriculture: building a global agricultural data and open innovation ... power of big data analytics through inspiring and innovative

Reuse: Price data

Page 35: CGIAR PLATFORM FOR BIG DATA IN AGRICULTURE · data in agriculture: building a global agricultural data and open innovation ... power of big data analytics through inspiring and innovative

Maximum profitability

Page 36: CGIAR PLATFORM FOR BIG DATA IN AGRICULTURE · data in agriculture: building a global agricultural data and open innovation ... power of big data analytics through inspiring and innovative

• Old data can be used for new science

• Empirical models will gain importance as more

data becomes available

• Interoperability for data aggregation

will become increasingly important

Page 37: CGIAR PLATFORM FOR BIG DATA IN AGRICULTURE · data in agriculture: building a global agricultural data and open innovation ... power of big data analytics through inspiring and innovative

Toward operational multi-scale

investment prioritisation

Julian Ramirez Villegas – CIAT

Page 38: CGIAR PLATFORM FOR BIG DATA IN AGRICULTURE · data in agriculture: building a global agricultural data and open innovation ... power of big data analytics through inspiring and innovative

Outline

1. Climate change –a wicked problem

2. What investments? Where? –data is

here to help

• Smallholder Adaptation Atlas

• Country CSA Investment Planning

• … and diving even deeper

3. Powered by CGLabs

Page 39: CGIAR PLATFORM FOR BIG DATA IN AGRICULTURE · data in agriculture: building a global agricultural data and open innovation ... power of big data analytics through inspiring and innovative
Page 40: CGIAR PLATFORM FOR BIG DATA IN AGRICULTURE · data in agriculture: building a global agricultural data and open innovation ... power of big data analytics through inspiring and innovative

Models of crops,

livestock and

socioeconomics

Impacts

Vulnerability

Adaptation

options

Investment

costs

What investments? Where? –data is here to help

Page 41: CGIAR PLATFORM FOR BIG DATA IN AGRICULTURE · data in agriculture: building a global agricultural data and open innovation ... power of big data analytics through inspiring and innovative

Smallholder Adaptation Atlas

Page 42: CGIAR PLATFORM FOR BIG DATA IN AGRICULTURE · data in agriculture: building a global agricultural data and open innovation ... power of big data analytics through inspiring and innovative

To support investment prioritization by

the global commission on adaptation

Smallholder Adaptation Atlas

Page 43: CGIAR PLATFORM FOR BIG DATA IN AGRICULTURE · data in agriculture: building a global agricultural data and open innovation ... power of big data analytics through inspiring and innovative

Data to support decisions

12Hazard

variables

5Climate

scenarios

33Crops

Heat stress

occurrence (future)

continental coverage

> 100k individual layers

180Adaptation

options

Page 44: CGIAR PLATFORM FOR BIG DATA IN AGRICULTURE · data in agriculture: building a global agricultural data and open innovation ... power of big data analytics through inspiring and innovative

AGRA Countries

Inorganic

inputs

Cons. Ag.

R&D

Page 45: CGIAR PLATFORM FOR BIG DATA IN AGRICULTURE · data in agriculture: building a global agricultural data and open innovation ... power of big data analytics through inspiring and innovative

Country CSA Investment Planning

Page 46: CGIAR PLATFORM FOR BIG DATA IN AGRICULTURE · data in agriculture: building a global agricultural data and open innovation ... power of big data analytics through inspiring and innovative

Frequency: Every other yearIntensity: severe (80/100)

System: rainfed maize-beanSensitivity: high (75/100)

Hazard characteristics

System characteristics

Risk scoring for selected hazard

Fre

qu

en

cy

Impact

Mild drought

Severe flooding

Lightning

Severe drought

Moderate heat stress on livestock

Waterlogging

Moderate heat stress

on crops

Earthquakes

Pollution

Understand climate hazards

Page 47: CGIAR PLATFORM FOR BIG DATA IN AGRICULTURE · data in agriculture: building a global agricultural data and open innovation ... power of big data analytics through inspiring and innovative

Assess and prioritize investments

Page 48: CGIAR PLATFORM FOR BIG DATA IN AGRICULTURE · data in agriculture: building a global agricultural data and open innovation ... power of big data analytics through inspiring and innovative

Powered by CGLabs

Page 49: CGIAR PLATFORM FOR BIG DATA IN AGRICULTURE · data in agriculture: building a global agricultural data and open innovation ... power of big data analytics through inspiring and innovative

Easy

to use

Customized and

customizable

Fast, reliable

tech support

CGIAR’s

data ready

Fast for

downloading

non-CGIAR data

Easily

scalable

Replicability, use in

teaching and

debugging / testing

Can share with

anyone, at minimal

transaction cost

Notebooks are

useful for capacity

building

Streamlining

analysis

Why CGLabs?

Page 50: CGIAR PLATFORM FOR BIG DATA IN AGRICULTURE · data in agriculture: building a global agricultural data and open innovation ... power of big data analytics through inspiring and innovative

bigdata.cgiar.org