Building an AI Credit-Scoring Engine at Payriff (Fintech Internship)

Over summer 2024 I worked at Payriff on an AI credit-scoring engine that fuses traditional bank data with nontraditional signals (mobile-provider behavior, ride-hailing driver earnings, and other alternative data), and shipped a RAG chatbot that teaches developers the Payriff API, cutting onboarding from hours to minutes. This is the internship I don't write about often enough. The work is proprietary, so the specifics below are generalized, but the architecture and lessons are the ones I'd give to anyone building AI for fintech onboarding.

TL;DR

• Credit-scoring engine: combined bank data with nontraditional signals (mobile-provider behavior, ride-hailing earnings, and more) to score thin-file applicants better.
• Developer chatbot: RAG over the public Payriff API docs; answers questions with code + doc links.
• Team dynamic: the wins came from pairing ML engineers with credit analysts, not from tuning models in isolation.
• Stack: Python, FastAPI, OpenAI, standard ML tooling. Nothing exotic; the value is in the data pipeline and the domain expertise.

Context

Payriff is a payments platform. The company wanted two AI-shaped problems solved:

1. Score applicants more accurately, especially "thin-file" customers without rich histories in a single bank.
2. Reduce developer onboarding friction for merchants integrating the Payriff API for the first time.

Both problems had been worked on with conventional software. Both had ceilings that AI could plausibly raise.

Problem 1 — credit scoring with nontraditional data

The core insight came from the credit analysts on the team, not from any ML paper: everyday behavioral data correlates with repayment reliability. Someone who has maintained the same phone plan for 18 months, tops up consistently, and hasn't switched devices in a year tends to repay on time. Someone with an erratic history often repays erratically, too. And mobile was only one source. The engine drew on a range of nontraditional signals, like earnings history for ride-hailing drivers (an Uber driver with steady weekly payouts has a repayment signal no bank statement shows).

The engine fuses data sources like these:

And produces a unified score. For thin-file applicants, where the bank signal alone is weak, the alternative signals carry real weight.

The modeling work was standard: gradient-boosted trees with interpretable features (no black-box deep nets for regulated decisions), rigorous calibration, careful train/validate/test splits with time-based holdouts. The interesting work was feature engineering in tight collaboration with the credit team. They knew which of these nontraditional signals correlated with their existing default portfolio better than any automatic feature selection would have surfaced in three months of tuning.

Auditability

Everything that goes into a score is logged:

• Every feature value (with provenance).
• Every model version the score came from (pinned, never "latest").
• Every manual override (with reason and reviewer).

In a regulated stack, interpretability isn't optional. It's the difference between shipping and not shipping.

Problem 2 — the developer-docs chatbot

Developers integrating Payriff were spending hours reading API docs. The docs weren't bad; the problem is that integration is inherently "I need to do X specifically," and docs are organized by endpoint, not by intent.

I built a RAG chatbot over the public Payriff API documentation so new integrators could ask questions in plain language and get back:

• A precise code snippet for their language.
• A link to the exact documentation section the answer came from.
• (When the retrieval was weak) an honest "I'm not sure — here's the closest docs section."

User: "how do I charge a saved card in Node?"
Bot:
  <Node snippet>
  Source: /docs/payments/charge-saved-card

Pipeline:

1. Embed the docs per heading chunk with OpenAI embeddings.
2. Retrieve top-K chunks for each incoming question.
3. Feed the LLM the question + retrieved chunks + a strict system prompt that requires a source link.
4. Fail loudly when retrieval confidence is below a threshold (better to say "I don't know" than to hallucinate a parameter name).

In the internal pilots we ran, first-day developer onboarding time dropped from "hours of doc-scanning" to "minutes of targeted Q&A."

What I learned

• The ML was the easy part. The data pipeline and the collaboration with non-ML domain experts were the hard part, and the part that made the system accurate.
• Thin-file scoring with alternative data is an emerging-markets superpower. It lets you make fair decisions for people a traditional bureau underrates.
• A docs chatbot is a conversion tool. For any platform with a developer audience, it's one of the highest-leverage AI products you can ship.
• Interpretability is a shipping requirement in regulated stacks: gradient-boosted trees and clear features over deep nets and mysterious embeddings.

Background / resume context

This was an AI Intern role at Payriff in Baku, Azerbaijan (June to September 2024). It's listed on my about page and projects page. For related work from my Purdue AI + Mathematics degree and other projects, see my blog archive and the handwriting-generator writeup.

Key takeaways

• Nontraditional data (mobile-provider behavior, ride-hailing earnings, and similar signals) is a strong complement to bank data for thin-file credit scoring.
• Pair ML engineers with domain experts. The credit analyst knows which features matter better than your AutoML pipeline does.
• A well-grounded RAG chatbot is the single most valuable AI feature for a developer-facing API platform.

References

• Payriff, a payments platform in Baku, Azerbaijan
• My projects · About me