Scaling Secure AI: Turning Enterprise Data into Impact (Behind The Build with Tharani Part 2)

It's not like building a front-end user

interface because it's pretty defined.

interface because it's pretty defined. Once you define it, the unknown is

Once you define it, the unknown is really like whether your user likes it

really like whether your user likes it or not or how they use it. But this is

or not or how they use it. But this is even at the core of implementation.

even at the core of implementation. There's a lot of variability and

There's a lot of variability and variation of quality and kind of

variation of quality and kind of reliability of that products itself. So

reliability of that products itself. So there's a lot of like iterative test

there's a lot of like iterative test that goes into

that goes into you're testing a lot of like uh it's

you're testing a lot of like uh it's very much a real R&D because you're

very much a real R&D because you're going through the process with like

going through the process with like first hypothesis this tech tech stack

first hypothesis this tech tech stack would result in this result or this

would result in this result or this outcome going through that exercise and

outcome going through that exercise and then once you do it you have a different

then once you do it you have a different result or it doesn't meet the needs of

result or it doesn't meet the needs of the the team. So you end up drawing

the the team. So you end up drawing board making changes and then coming

board making changes and then coming back again. So having done the proof of

back again. So having done the proof of concept and and you kind of have the

concept and and you kind of have the tech stack you understand what sort of

tech stack you understand what sort of inputs it needs to go in to get the

inputs it needs to go in to get the outputs and the infrastructure that

outputs and the infrastructure that needs to support it at a very small test

needs to support it at a very small test kind of package. Now what goes into

kind of package. Now what goes into production? How does how do you go from

production? How does how do you go from let's say having one or two PC's that's

let's say having one or two PC's that's getting approved by the team going okay

getting approved by the team going okay this results these results are pretty

this results these results are pretty good. What happens next?

good. What happens next? Okay. So once we are done with the P and

Okay. So once we are done with the P and the team is satisfied with the results

the team is satisfied with the results and we made the other the progress of

and we made the other the progress of involving the other features into the

involving the other features into the PC. So not into the system. So when we

PC. So not into the system. So when we are implementing PC we just uh took the

are implementing PC we just uh took the prioritized features but in the real

prioritized features but in the real system there will be other adjoining

system there will be other adjoining features as well that support that

features as well that support that specific feature. So we figured it out

specific feature. So we figured it out those kind of things and we figured it

those kind of things and we figured it out the technologies that we'll be using

out the technologies that we'll be using that we can use to implement those and

that we can use to implement those and we came up with the architecture diagram

we came up with the architecture diagram of the system and the technical

of the system and the technical architecture document for those systems

architecture document for those systems and we send those documents uh to the

and we send those documents uh to the team for the feedback and yeah it was

team for the feedback and yeah it was the progress. Okay,

the progress. Okay, you need to do all of this because it's

you need to do all of this because it's a pretty big company, right? It's has a

a pretty big company, right? It's has a pretty large revenue.

pretty large revenue. Yeah.

Yeah. Um I think it roughly has like 100

Um I think it roughly has like 100 something engineers. So lots of product

something engineers. So lots of product that are being simultaneously being

that are being simultaneously being built. How does atomic team actually

built. How does atomic team actually play well with all of that? Because

play well with all of that? Because there's a lot going on. It's a big

there's a lot going on. It's a big machine that's running already and as an

machine that's running already and as an outsider in with very short context

outsider in with very short context you're kind of coming in. There's a lot

you're kind of coming in. There's a lot of back and forth is what I'm what I'm

of back and forth is what I'm what I'm understanding, right? and you're having

understanding, right? and you're having to review and make sure that those those

to review and make sure that those those uh changes that you're making doesn't do

uh changes that you're making doesn't do anything negative to the existing system

anything negative to the existing system and then also you have to build it in a

and then also you have to build it in a safe way that it uh respects all the

safe way that it uh respects all the sort of the safety protocols that they

sort of the safety protocols that they have in place. Are there any um kind of

have in place. Are there any um kind of training you need to go through? Are

training you need to go through? Are there anything that you had to do to uh

there anything that you had to do to uh get yourself um approved uh to be able

get yourself um approved uh to be able to work within their environment? Was

to work within their environment? Was there anything that you had to do that

there anything that you had to do that way?

way? So they have a very secure environment.

So they have a very secure environment. So we went through a lot of security

So we went through a lot of security training they provided and on the other

training they provided and on the other hand so when we are providing a solution

hand so when we are providing a solution for the problem. So at the first there

for the problem. So at the first there was some hesitation from the team

was some hesitation from the team because AI is new and the people are not

because AI is new and the people are not much familiar with the AI and the

much familiar with the AI and the results and in this sensitive kind of

results and in this sensitive kind of environment. So most of the people are

environment. So most of the people are uh worry about the sensitivity of the

uh worry about the sensitivity of the data. So we were thinking to uh explain

data. So we were thinking to uh explain about what we are doing and how we can

about what we are doing and how we can handle the sensitivity of the data and a

handle the sensitivity of the data and a lot of things. So we have backto-back

lot of things. So we have backto-back meetings with the team and we wrote a

meetings with the team and we wrote a lot of documents explaining what we can

lot of documents explaining what we can do to secure the data and what the AI

do to secure the data and what the AI model will consume in sake of the in

model will consume in sake of the in sake of implementing the system and all.

sake of implementing the system and all. So it was a tough progress to explain

So it was a tough progress to explain the things but we were finally

the things but we were finally successfully achieve the thing and we

successfully achieve the thing and we were able to convince them about using

were able to convince them about using the AI in their project successfully.

the AI in their project successfully. Yeah.

Yeah. Yeah. It's it's pretty scary right like

Yeah. It's it's pretty scary right like all the data that you used to have

all the data that you used to have at least in the context of

at least in the context of what people understand about AI

what people understand about AI especially if you're now thinking about

especially if you're now thinking about like you know plugging into an API

like you know plugging into an API that's actually just calling an external

that's actually just calling an external service. uh that's very risky and that's

service. uh that's very risky and that's not exactly the implementation you did

not exactly the implementation you did because obviously for the safety reasons

because obviously for the safety reasons you end up building it internally had a

you end up building it internally had a lot of controls uh but that's what

lot of controls uh but that's what people understand when it comes to AI

people understand when it comes to AI implementation and and it seems very

implementation and and it seems very scary what helped them you said a lot of

scary what helped them you said a lot of documentation that helped them to

documentation that helped them to understand was there anything else that

understand was there anything else that you had to show for or do that that

you had to show for or do that that helped them be more comfortable build

helped them be more comfortable build the trust with the solution that we were

the trust with the solution that we were that you were building

that you were building yeah so we did one good implementation

yeah so we did one good implementation That's the MCP server. So for the AI

That's the MCP server. So for the AI agents, we have to pull up the data that

agents, we have to pull up the data that need to be processing. So for example,

need to be processing. So for example, if we have to build some advising

if we have to build some advising questions for the students. So we have

questions for the students. So we have to get the data related to student and

to get the data related to student and send it to AI model to process or

send it to AI model to process or generate a question. So there was a

generate a question. So there was a concern about pulling the data from

concern about pulling the data from their internal database because it's

their internal database because it's very sensitive and not. So we came up

very sensitive and not. So we came up with a solution of implementing the MCP

with a solution of implementing the MCP server which which is the model context

server which which is the model context protocol of providing what exactly the

protocol of providing what exactly the AI system need for the processing or the

AI system need for the processing or the for the system. So we built it built

for the system. So we built it built this server by using the APIs they have

this server by using the APIs they have given to fetch the data from their

given to fetch the data from their database and we built specific tools

database and we built specific tools that we need for the purpose of the

that we need for the purpose of the agents. So the agents now will get only

agents. So the agents now will get only the data they need nothing less or

the data they need nothing less or nothing more. So we are sending the

nothing more. So we are sending the appropriate structured data to the AI

appropriate structured data to the AI agent through this MCP server. So it's

agent through this MCP server. So it's very convenient and it's very secure

very convenient and it's very secure also. Yeah,

also. Yeah, that's very interesting to hear because

that's very interesting to hear because MCP is the all the rage right now across

MCP is the all the rage right now across all platforms. every single product

all platforms. every single product that's has any like external API they're

that's has any like external API they're making that whole function uh enabled on

making that whole function uh enabled on MCP so that all LM models can access it.

MCP so that all LM models can access it. to give context to build API

to give context to build API integration. It's custom. It's custom

integration. It's custom. It's custom integration for two applications to

integration for two applications to communicate with each other. With MCP,

communicate with each other. With MCP, it's kind of a universal plug-and-play

it's kind of a universal plug-and-play where you have kind of a set standard

where you have kind of a set standard protocol and then and that defined

protocol and then and that defined protocol is a kind of uni universally

protocol is a kind of uni universally accept acceptable

accept acceptable now by all the models that are out there

now by all the models that are out there that wants to integrate with them. So

that wants to integrate with them. So this approach that you went about doing

this approach that you went about doing it's actually for internal use case.

it's actually for internal use case. It's not even external. So all the data

It's not even external. So all the data endpoints that you're building, it's

endpoints that you're building, it's actually for within the platform of this

actually for within the platform of this company.

company. It is not going outside of it. And the

It is not going outside of it. And the benefit here is actually the controls

benefit here is actually the controls you have uh and and it's much safer

you have uh and and it's much safer approach.

approach. And also for the other AI agents we'll

And also for the other AI agents we'll be building in feature, we can use the

be building in feature, we can use the same NCP server to fetch all the

same NCP server to fetch all the necessary information we need. So we

necessary information we need. So we don't need to build separate server or

don't need to build separate server or separate APIs for each of the AI agents

separate APIs for each of the AI agents but we can use the same MCP server and

but we can use the same MCP server and in future future if we need more tools

in future future if we need more tools or if we need more data we can add more

or if we need more data we can add more tools to the MCP server that can serve

tools to the MCP server that can serve the purpose for the other agents.

the purpose for the other agents. So build once and deploy across

So build once and deploy across that's amazing. So that's like the

that's amazing. So that's like the biggest unlock for internal application.

biggest unlock for internal application. how you can kind of scale very quickly

how you can kind of scale very quickly without having to do these individual

without having to do these individual connections that you were you would

connections that you were you would normally have to do otherwise.

normally have to do otherwise. Yeah.

Yeah. Okay. Amazing. Um is there a lesson that

Okay. Amazing. Um is there a lesson that you had across this whole project that

you had across this whole project that you want to share

you want to share when we are dealing with the AI? So we

when we are dealing with the AI? So we always make sure about the security of

always make sure about the security of the data as the first point and we have

the data as the first point and we have to send only the necessary information

to send only the necessary information to the AI without sending nothing more

to the AI without sending nothing more or nothing less to the concept and the

or nothing less to the concept and the other one is the structure of the AI

other one is the structure of the AI structure of the data we are sending to

structure of the data we are sending to the AI. But if we are sending an

the AI. But if we are sending an unstructured data or semistructured

unstructured data or semistructured data, there will be a lot of problems

data, there will be a lot of problems like hallucination and uh the uh not not

like hallucination and uh the uh not not uh the quality of the output won't be

uh the quality of the output won't be that much good as we expect. So we

that much good as we expect. So we always consider about the structure of

always consider about the structure of the data we are sending and always

the data we are sending and always consider about good prompt because we

consider about good prompt because we have to make sure about the prompt

have to make sure about the prompt engineering and we have to use the right

engineering and we have to use the right tools to create the prompts and u make

tools to create the prompts and u make the AI to work more efficiently.

the AI to work more efficiently. So it goes we'll go on to our next

So it goes we'll go on to our next section which is the lightning round. So

section which is the lightning round. So you just take a second to respond. Not

you just take a second to respond. Not not much more explanation is needed. We

not much more explanation is needed. We have about few questions we just want to

have about few questions we just want to run through. So one piece of tech that

run through. So one piece of tech that you can't live without.

you can't live without. It's tough. Um at the current point I

It's tough. Um at the current point I can say AI tools because even with the

can say AI tools because even with the work and for my personal use I'm using a

work and for my personal use I'm using a lot of AI tools specifically uh for to

lot of AI tools specifically uh for to create the documentation or to

create the documentation or to brainstorm what I'm thinking related to

brainstorm what I'm thinking related to work and there are a lot of technologies

work and there are a lot of technologies that are emerging. So instead of going

that are emerging. So instead of going through the articles and all I usually

through the articles and all I usually go to the AI and did a brain quick

go to the AI and did a brain quick brainstorm about what I'm thinking and

brainstorm about what I'm thinking and what's the solution that can be useful

what's the solution that can be useful by using these new emerging tools and

by using these new emerging tools and all. So there are a lot more to come but

all. So there are a lot more to come but I am currently very much excited about

I am currently very much excited about this thing. Yeah.

this thing. Yeah. What's a tech trend that you're excited

What's a tech trend that you're excited about right now? If I'm considering

about right now? If I'm considering anything related to AI, maybe I am more

anything related to AI, maybe I am more excited about the agentic AI system

excited about the agentic AI system because from the articles I read, it's

because from the articles I read, it's more about like a human like we think

more about like a human like we think the AI can alo also think. So they can

the AI can alo also think. So they can act according to the context they can

act according to the context they can derive to a solution based on the

derive to a solution based on the specific goals and all. So I'm more

specific goals and all. So I'm more excited how it will work and also more

excited how it will work and also more excited how we are going to build it.

excited how we are going to build it. What is a underrated skill for a

What is a underrated skill for a software developer?

software developer? I would say writing a good documentation

I would say writing a good documentation because when we are rushing with the

because when we are rushing with the work we didn't uh know about the value

work we didn't uh know about the value of good documentation but if we have a

of good documentation but if we have a document good documentation it will be

document good documentation it will be useful to onboard new stuffs and also it

useful to onboard new stuffs and also it will be useful if we want to revise

will be useful if we want to revise something that we did past. So it's very

something that we did past. So it's very helpful and also if you learn any new

helpful and also if you learn any new things it good it's good if we document

things it good it's good if we document it because instead of going around in

it because instead of going around in the web to search about the new things

the web to search about the new things again it's good if we have the already

again it's good if we have the already taken notes we can really grab the

taken notes we can really grab the things yeah

things yeah where do you document your thoughts

where do you document your thoughts so mostly I'm using the Google docs

so mostly I'm using the Google docs because I can invite the other team

because I can invite the other team members to put up their thoughts through

members to put up their thoughts through the collaborations and all also we can

the collaborations and all also we can comment the other documents as

comment the other documents as So for me, Google Docs is a helpful one.

So for me, Google Docs is a helpful one. Yeah.

Yeah. And final closing thought to the

And final closing thought to the developers out there who are still at

developers out there who are still at the edge of thinking about experimenting

the edge of thinking about experimenting with AR, curious, they might they might

with AR, curious, they might they might be even listening to this podcast just

be even listening to this podcast just so that they can get uh kind of used to

so that they can get uh kind of used to the concept and understand how to go

the concept and understand how to go about doing it. What advice would you

about doing it. What advice would you give them? So I would stay I would say

give them? So I would stay I would say start by curious like put your interest

start by curious like put your interest on AI and start by working not just by

on AI and start by working not just by reading but also with some hands-on

reading but also with some hands-on exercise. So if you came across any new

exercise. So if you came across any new concept just put some rough coding and

concept just put some rough coding and play around with the c code and all they

play around with the c code and all they have given in the documentation and also

have given in the documentation and also always treat AI as your companion not as

always treat AI as your companion not as a competition because with the help of

a competition because with the help of AI you will be able to build your

AI you will be able to build your expertise a lot and you'll be able to

expertise a lot and you'll be able to shine in your work.

shine in your work. I love that. Don't uh have I as your

I love that. Don't uh have I as your competition but as you

competition but as you Yeah,

Yeah, I like that. Awesome. Thank you so much

I like that. Awesome. Thank you so much for your time. authing

and kind of uh I think I think really

exciting in terms of where everything is

exciting in terms of where everything is headed especially with the type of work

headed especially with the type of work that we're doing atomic as well. So

that we're doing atomic as well. So thank you so much for sharing your

thank you so much for sharing your thoughts and your experience. We'll talk

thoughts and your experience. We'll talk to you soon.

to you soon. Thank you so much.

Thank you so much. If today's story resonated with you I'd

If today's story resonated with you I'd love to hear from you. Who should we

love to hear from you. Who should we bring next on our behindthebuild series?

bring next on our behindthebuild series? Send us your suggestions at btbat

Send us your suggestions at btbat atomicbuild.com or connect with me on

atomicbuild.com or connect with me on LinkedIn. We're always on the lookout

LinkedIn. We're always on the lookout for the next great building story to

for the next great building story to share. And if you're facing your own

share. And if you're facing your own technology challenges, whether it's AI

technology challenges, whether it's AI exploration, custom development, or

exploration, custom development, or scaling your own platform, let's talk.

scaling your own platform, let's talk. At Atomic Build, we specialize in this

At Atomic Build, we specialize in this space. And thanks for joining us on

space. And thanks for joining us on Behind the Build. Until next time, keep

Behind the Build. Until next time, keep building.