The Government of Ireland’s Department of Agriculture, Food, and the Marine (DAFM) processes grant applications submitted by farmers and their agents. However, incorrect document submissions by applicants led to data breaches in violation of the EU’s General Data Protection Regulation (GDPR). DAFM worked with Red Hat and Version 1 Software to create SmartText, a machine learning (ML) text analysis platform that scans documents and images for sensitive information and prevents unauthorized access. With a foundation built using Red Hat container and management technology, DAFM can correctly categorize documents while protecting back-end systems and delivering new features faster.
Benefits
- Significantly reduced GDPR breaches with intelligent, AI/ML text analysis solution
- Reduced development time from weeks to days with automation, APIs, and DevOps
- Improved security and stability with enterprise technology and support
Preventing GDPR violations and data breaches
The Government of Ireland’s Department of Agriculture, Food, and the Marine (DAFM) leads, develops, and regulates the country’s agrifood sector, including farming, seafood, forestry, plant health, and animal services. To meet its mission of providing innovative, sustainable services, DAFM aligns with the country’s Public Service ICT Strategy to provide digital services to citizens, businesses, and government agencies.
DAFM’s Basic Payments Scheme (BPS) handles applications and payments for farming funding grants. It is responsible for issuing €1.2 billion yearly to 120,000 farmers, in line with the European Union (EU)’s General Data Protection Regulation (GDPR) that protects the personal data and privacy of EU citizens. Each year, BPS receives 30,000-40,000 grant application documents submitted to its website by farmers or their agents. These documents often include Personally Sensitive Information (PSI), such as birth certificates, legal documentation, or medical information. However, farmers often failed to correctly classify documents as containing PSI. Agents—some of whom serve dozens of farmers—sometimes uploaded documents to the wrong application. As a result, highly sensitive personal data was exposed to BPS employees or even other farmers.
To mitigate the risk of financial penalties and reputational damage from major GDPR breaches, DAFM sought to replace its legacy data system and processes with an intelligent solution that would help correctly identify PSI and automate detection of breaches.
“Their previous system matched PSI with document numbers, but there was no way to search within documents,” said Stephen Griffin, Architect and Software Development Manager at Version 1 Software, DAFM’s local partner. “Processes for reporting and correction of possible PSI exposure were manual, but with the sheer volume of documents received, constant human review was impossible.”
Building a foundation for machine learning text analysis with Red Hat
BPS worked with Version 1 Software to outline requirements for an intelligent, automated approach to grant application processing. After evaluating several vendors, DAFM and Version 1 decided to use Red Hat OpenShift, supported by Red Hat 3scale API Management and Red Hat’s single sign-on (SSO) technology.
“Our research showed that we needed an enterprise Kubernetes platform to create our machine learning text analysis solution. Red Hat OpenShift was the clear leader in that market. Red Hat also provides the security and control we need as a government organization,” said Gareth Sheerin, Enterprise Architect, Department of Agriculture, Food, and the Marine.
With guidance from Version 1 and Red Hat Consulting, DAFM created SmartText, an ML text analytics solution that can extract metadata from submitted documents to correctly identify PSI, in just a few weeks. To meet EU requirements for data ownership, this solution runs on Red Hat OpenShift on premise in DAFM’s datacenter. 3scale API Management, part of Red Hat Integration, uses application programming interfaces (APIs) to connect the platform with DAFM’s citizen-facing application. Red Hat’s SSO technology supports protected mobile access for applicants using OpenID Connect and OAuth authentication.
“SmartText is now a key part of our Smart Action Suite, a set of enterprise productivity applications focused on addressing common service and technology challenges,” said Sheerin.
Its success using open source technology to create the SmartText solution led to DAFM’s recognition with a 2021 Red Hat Innovation Award.
Automating and simplifying compliance to protect farmer data
Reduced GDPR breaches with intelligent text analysis
SmartText, hosted on Red Hat OpenShift, uses real-time artificial intelligence (AI) and ML capabilities to extract metadata and other contextual information from unstructured grant application documents. These documents, including scans of handwritten or typed letters, are analyzed for sentiment, semantically similar words, topics, and entity names.
“In both written text and images, we look for information pertaining to minors, medical conditions, even credit card numbers or what we call a herd number, which is personally identifiable to an individual,” said Griffin.
Access to the SmartText service itself is protected through 3scale API Management. As a result, DAFM can drastically reduce the number of GDPR breaches—and the associated reputational damage and financial penalties—each year.
