Welcome to the fourth of a series of webinars going through 51Degrees’ various real-time data products.

Today we're going to be going through Java for both device detection and IP intelligence.

I'm joined by Eugene Dorfman, who is one of the developers that works on the 51Degrees codebase. Welcome to the webinar, Eugene.

Thank you for introduction.

So last time we covered C#, we're now going to go through Java.

But we're not going to assume you've seen the previous webinar.

This one is very much for the Java developers.

So let's make a start, Eugene.

Let's go through to start with the 51Degrees codebase and where the examples that you're going to see today can be found.

Right, so 51Degrees has several products, device detection, IP intelligence, and we have various assets on GitHub in the open source that demonstrate how these products work.

So the primary ones for Java would be IP intelligence Java examples repository, and the similar repository for device detection.

So it will be device detection, Java examples. So everything you need to run to experiment with 51Degrees products will primarily live in those repositories.

And it's got lots of documentation on how to run and tweak those examples and we'll show you them running in a minute.

So which one would you like to start with?

So let's start with IP intelligence on the premise.

Okay.

So we've got 2 examples.

We have the console example and the web example.

There are various other examples as well that we're not going to look at today.

We encourage developers to start with the console example because it goes through all the concepts in one simple file.

So why don't we start there?

Right.

So, let me first run it and then we'll explain the concepts that are forming this example. So, if I run this example, we're gonna see an output that contains the input value, which is an IP address, and the properties that we have retrieved for this IP address, from the data file.

So, this example is actually using the data file, and the 51Degrees library retrieves those properties for this particular IP address from the data file.

the example itself is built around the concept, we call pipeline.

Well, it is pipeline, in essence, because it's a chain of elements that process data.

They take data as input, and they produce the data as output.

And in the code, pipeline is reflected as a pipeline object, actually.

So, you construct it using, this line of code, well, actually, it's split into 2 lines, but, essentially, it's one line of code, which allows you to take the pipeline configuration file and construct the pipeline.

So pipeline consists of elements.

Each element is processing data.

So, here is how the configuration file looks like.

So, for command line, we have just one element in this pipeline for IP intelligence, which is signified by its builder name, which is the IP intelligence on premise engine.

So, this engine has various configuration parameters, you can use them in code to construct the engine, or you can use them through configuration, which is more convenient.

And the most important parameter here is the data file.

By default, we are using a light data file, but in the example code, we override it with the...

It's just a pause at that point.

So there are broadly two types of data file for both IP intelligence and device detection.

You've got the free light data file, which is contained in GitHub, and then you've got the subscription data file, often called Enterprise, which contains more properties.

It's updated daily, etcetera.

So most of our examples are written to perform basic functionality using light, so you can actually see what's going on.

But if you want up to date data and things like that, then you'd need to look at getting the enterprise and you can contact us for a subscription.

Right.

And this example is actually constructed to take the data file as an online parameter.

So we're overriding configuration.

We've got a command line, right?

Yeah, just for convenience.

This is.

And in the analyse evidence function here, you can see how the input data, which we call evidence, is consumed, or is supplied to the pipeline, and it goes through the pipeline, when we call this process method.

And we get the output in this line, where we get, we call data get, data don't get, and, which is output various IP intelligence properties.

So the same, the same concept, so the object that is, like, the transient object that we operate with is called flow data.

So, we use, we call the pipeline that create flow data to obtain this transient object, it will be done on every request, so it's a very lightweight object, the sole purpose of this object is to carry data, through the pipeline.

And we operate using on this flow data object.

So we add evidence to it, we process, we get results from it.

So this is the same concept across both, across all products of 51Degrees.

We also have this example running.

For web, so this is a command line example, we can switch to look.

Well, we've just shown that so in the task window there you've got the IP addresses and then these are ones we've included in the example and then some data coming back for them.

But of course, that's not very visual.

So yeah, let's have a look at the web version.

Right.

So in order to not hard code things let's try to switch to web.

So we have this hierarchy of examples, we have like console and web examples, so we are now going to switch to...

We're just gonna use on-premise to start with.

We can come back to combining device detection and IP intelligence shortly.

Right.

So we're gonna use getting started on-prem, which is a mixed, which is IP Intelligence only.

So, it also uses the configuration.

We can look at the configuration file.

Basically, it's just this single engine as in the console example.

So, if I switch to Web getting started.

So many examples to choose from.

Right.

So I believe this one is using still AD82, we didn't replace that port number, but that's fine.

So this is running jetty on 8082 on your death machine, yeah?

Yeah, so we are gonna see on 8082.

So.

Okay, cool.

So we're now getting data coming back for a local host.

Right. Look back.

It's not so much of useful information.

Not very internet.

Yeah.

But if we kind of hardcore, oh, well, supply the evidence, we are going to get some results.

For example, this is a Swedish IP address.

But more interesting would be to run it through.

Well, just before you move on, if we could scroll down.

What you're seeing there is that area code coming back, which is a series of latitudes and longitudes in WKP, which is a WKT, sorry, which is a geospatial format, and then we've rendered that on a map, so you can actually see that particular IP addresses covering mainly Sweden.

Right.

So, yeah, so let's see how we can run it from the outside so I can, connect and see what what's relevant from my IP address right now.

So, for that, we are gonna switch the reverse proxy to use port AD82.

So this is, we can connect to a live web address.

Yeah, and this is our IP address.

This is where we are located right now.

Cool. About that place.

And this is the area that we are currently covering.

So what you're seeing here is, yes, there's an accuracy, there's a latitude and longitude, there's a radius, you could render this as a circle, but actually, a series of polygons is more important, to give you a true representation of where that device might be, with carrier grade gnat, cellular networks, these kind of things.

It's very rare that an IP address is going to be precisely located these days.

So working with that WKT, geographic area gives you an advantage.

right.

So, we've seen AP intelligence, getting started examples, come online.

And that was all on premise, right?

This is all on premise, right?

Right, and it's all running in the process on your development machine.

There's no external servers or any of that kind of stuff.

Correct.

So it uses the data file that we supply it through command line.

I had it pre-downloaded, so we don't spend time, download it into gigabytes.

Just data file.

Very cool.

So all that start, everything you saw, that it was the startup time, getting it ready for action, that pipeline initialising with that engine in place, very, very quick indeed.

right, so what do we want to see next?

Well, let's have a look at device detection.

Okay.

So, while Eugene is switching over… we have the same sort of examples for device detection.

It's a little bit more involved because there's more than just an IP address that's needed as evidence.

So we're going to show you that working in isolation before we then move on to how we can combine the two together.

Right.

So, I have opened the device detection Java examples repository and we have several dimensions to those examples.

So, device detection allows you to, what we have, obviously, console and web dimension, of example, so we can show you either console or web examples, but we also can show you the example that works against the cloud API.

Okay, let's start with cloud first then if we can, and then we'll switch to on-premise.

Yeah, so there is some...

This 51Degrees cloud service that actually provides all the same data that you can use from the data file on premise, but it provides you through the JSON API.

And all you need to, to use it, you need the resource key, you need to obtain the resource key or the configurator.

I have done it briefly.

Just show people the configurator just so they see the entry point here.

So this is where you select your properties and configure your options.

We're not going to go through all of that now, but we've done that beforehand, but do have a play with that.

It's three steps.

Yeah, it takes you a minute at most if once you've selected your properties to get what you need.

Right, so for this, getting started, on-premise, this is a command line example that is going to...

Yeah, sorry, yeah.

Getting started cloud example.

So this is a command line example that is going to use the cloud service.

So, We actually supplied the resource key, I believe, in the command line.

Yes, we supplied it in the command line as a parameter.

So the pipeline that it has.

Let me actually check.

So it actually doesn't use the configuration file to create the pipeline.

It uses the special convenience, code object builder that allows to create just the same pipeline.

We could use a configuration file to add a device detection engine, but we only need, you know, we are using the command line here, so we only need device detection.

So we are not using the XML file.

We are just constructing a single pipeline with the single element, which is a device detection engine, and it's just simpler for a command line to use this API rather than...

So just to make sure it's sort of really understood by the viewers, you can configure these pipelines from an XML configuration in Java, or you can build them using code as well.

So for DevOps, people where you're more in that environment, you might prefer configuration.

If you're not going to change things very often or you've got a very simple pipeline, then you can just use code like in this example.

Right.

Or if you're, you know, you have some smart setup, for example, several pipelines, you might configure some of them through COVID and some of them through. As many pipelines as you like.

Or you can have one big pipeline.

Right.

So let's run this one real quick.

So, what it does… so, this is the input values, this is the evidence that we provide, and this is the output that it gets, from the request that it sends, from the response from the cloud.

So it sends a request to the cloud, gets the results and displays them here.

And, yeah, so for various….

So for example, here we have.

So this is evidence, right?

Input values is evidence.

Yeah, intel is evidence, and, well, results are results.

So we can supply, however, many HDP headers we want as evidence, and it will just be processed by the pipeline.

But I think this is not very visual, we can try to...

But again, this is the place to start for device detection.

So, you know, start with this console.

You'll see the different concepts being demonstrated, it could be a really good understanding of that creating the pipeline, the different options available there, and then how to consume it, providing evidence and getting the results.

But as you say, it's not very visual, and we're not using real devices.

So why don't we switch over to the web example?

Yeah, so we have this getting started, web, cloud, example.

So, with which kind of dimensions.

So this is still, it's gonna use the cloud service.

But it's gonna be a web server that we are running locally.

So if I, if I go into, sorry, before I show you that if I go into web, and go into getting started web cloud, we're gonna see that here we actually use configuration.

We use XML configuration for the pipeline, because we have more than one element.

And the pipeline is much easier to add them through XML and supply parameters to them rather than doing it in code.

Although there are convenience methods that allow you to do it in code as well.

So, we have several elements here, one of which is cloud request engine, that goes to the cloud endpoint, that we mentioned, it uses the resource key.

I believe we supply that resource key, as a command line parameter here, it will override this test resource key here in the configuration.

And then we have devices action cloud engine that just picks up the return properties, property values from, from the cloud and packs them conveniently into the data object.

And then we have two more JSON and trial script builder elements.

These are needed for client side evidence extraction and we'll get to it once we demonstrate how it works for mobile devices.

Very cool.

So you can see a slightly more complex configuration there.

Right, so, I believe this example runs on the local host 881, so you can see the web integration cloud example.

Yep.

So this is my Mac, MacBook Pro device, so you can see the tech split platform version and This is the, evidence that we send to, to the cloud and, yeah, basically this is the, the, so we're working from these HTTP headers, basically.

Right.

Yeah, so this is what we supply. As evidence.

So let's see.

I was actually, getting connect to your iPhone?

Yeah, yeah.

So let me do the iPhone mirroring.

for that, we will need to actually switch the port to 881, because through iPhone, I have to connect.

Let Okay, so there is a previous example.

From device detection, let me refresh the page so now we are looking at, sorry, we're looking at web integration, cloud example.

And as you see, my phone is detected as a iPhone 12 Pro Max, and, well, this is a group model.

So, the highest probability for the iPhone is the highest probability for the model is iPhone 12 Pro Max.

So this is happening through the JavaScript element that you showed us earlier in configuration, right?

I don't know if it's worth just going back and looking at that.

So what we've got is the HTTP headers that are helping us resolve the model, and then we've also got some JavaScript that's running, which is helping us narrow down the model.

So if we didn't have that running, it would be all possible iPhones, because Apple go out of their way to try and hide the model.

What we can do is narrow it down.

Right.

So, I think before this demo, we agreed to add one more property.

It's not yet added to the example.

I think it's a prize band.

Yeah we've edited it.

We edit it.

It's there.

Maybe in the example, if you get it off the website.

Yeah, so...

Yeah, yeah, yeah.

So it's already visible here.

So I just wanted to highlight how you would do this.

So, in the, well, actually, in the, in this, in this particular example, we don't list, properties that we want to get from, from cloud.

We would, otherwise, we would supply a list of properties that we want to get from cloud.

Here, we are actually getting all possible properties, which is maybe suboptimal for the...

Well, I have to correct you very slightly there.

So in that configuration page, we showed you earlier, where you get your resource key.

You can choose the properties that are returned at that point.

Yeah, that's right.

So you can basically choose the properties when you get your resource key, and you can also override them with the feature that you do.

Yes, yes, because this resource key that I'm particularly using for this example, it's actually a very heavy resource key.

It's got all possible properties of 51Degrees, just for demo purposes.

But, I mean, you can configure just a subset, you know, just, like, 3 properties here.

So you can limit it both ways.

But yeah, let's show people where they, how they consume these properties in code.

right.

So in the in this particular web example, we have .

added the .

.

To post.

Yeah, so we go to this function do device data.

This is where we add things on page.

So, for example, if I remove this, Just for the sake of the demo.

Um.

And we refresh. , we're not gonna have prize band here.

And to kind of add it back.

I would .

, I would add a Prize band and then this device object that we obtained earlier, as I, it's similar to how we obtained it from, from in the console application, from flow data, we, just ask for the Prize band.

This is a strongly typed accessor, so we have to do some, you know, to try, we wrap it with several methods.

We wrap it with the tri get method.

This method actually checks whether there is a value.

If there is no value, it will display an error message.

But then we have to display it as a string.

So it's not a property object.

It's a string.

So, that's why this wrapping happens here.

And then if we run, We are gonna get the price band for the iPhone 12 Pro Max.

This is the price of the iPhone, I believe, when it was first.

To launch price in US dollars, yeah.

So we don't track them over time, but it does give you an idea of the relative price of different devices.

right.

Okay, so this is device detection for the cloud device detection, but we also have the one premise device detection.

Yes, let's switch over.

So we're now going to go through the same output, but rather than the web server, calling the 51Degrees cloud service.

We're going to have it use the data file on premise within the same application space.

But everything else is exactly the same.

And you're going to see pretty much exactly the same code.

So we can really, really easily switch between using a web service, like a cloud as the back end, or the 51Degrees data file, which you can then run in process on premise.

Right, so it uses the common methods, actually, for both cloud and on premise examples on web.

But let's first maybe around the command line, the console example for...

Oh, yes.

Start with on premise console, yeah.

Right.

So, under a council, under, you have examples, we get, we have getting started on prem.

So this example is console getting started on prem.

I believe on the command line, I supply the, the data file to it.

Yeah, so this is one you downloaded for.

this one might be pretty old, but we'll check now, so I actually have a fresher data file available, but we'll, let's first run it and see what, what is the output.

So, here, you can see, like, various evidence, supplied as an example to the engine and engine outputs the results for this particular device, so we recognise this is an Android phone.

similarly here, well, for.

It's pretty much the same example, isn't it?

It's the same evidence.

So we're using the evidence for both examples, actually.

Right.

And I think this data file is very old.

It's from the December.

It's from December.

So I, actually can, well, I have a fresher data file in the, on my, on my disk right now.

So we'll run an example and see the date of that, fresher data file, if I...

Oh, if I were around this.

Um...

Maybe I missed something.

The path.

Yeah.

Johnny Greenstick.

Is it 51Degrees twice in the phone?

51 G repos?

Oh, okay.

So it's for council getting started on pram.

So right?

let me see.

Let me check if the file exists, actually.

Yeah, looks like it exists.

Is it permissions?

I cannot find.

And.

Okay, so it doesn't take the it doesn't like us providing the absolute path.

It wants the relative path for some reason, but we can.

Uh.

I think this is a permissions issue around Java, so one of the common support case problems we we have is actually this exactly this so being able to access the disk and then update the data file and come onto that in a minute.

So yeah, let's copy it across into the, to get it into the actual location where it's convenient to pick up for...

So we'll just copy it over and then we will just be able to change the name.

We will tell it to take it from Dubai's detection.. Detection data.

And hopefully, it will be picked up.

Yeah, so it was picked up, and you can see that it's a much newer data file from March 17th, so, but...

I'm a bit old, but it's a bit bit new.

I didn't download it.

Today, it was, like, last week when I was preparing, but, anyway.

Very cool.

So, you can see the different data files.

We've unexpectedly shown you a common support case there around permissions, which does tend to affect the Java environment a little bit more than other environments.

And you can see the different configurations and ways of loading it.

So we've got a slightly more complicated engine and pipeline builder here, haven't we, than the one we showed us?

Oh, sorry, it's, it's, as we used to explain, it's a dot net.

Sorry.

I, well, I mean, the, the, the complexity here, I guess is only in providing the correct data file, which is not real, not really a problem.

We can check how this pipeline is actually constructed, so it's constructed from this simple XML.

file.

We are overriding the data file that we buy default, we are we are using the light data file.

So if I didn't provide the data file on the command line, then the light data file would have been used.

we have performance profile here specified, which will touch on, closer to the end of this session.

so, yeah.

So let's try on premise with the web.

Yeah, finish off our 4 different demos.

Right, so for web.

Very similar to the cloud example, but it's gonna use the data file.

In fact, all the code, once you've created your pipeline in this instance, the only difference being the engine that's used to perform the device detection is exactly the same.

So you'll see there's a lot of common reuse across these examples.

Right.

So for Java, we have, well, actually, if you watch the previous webinar, you would see the JSON configuration files, but for Java, we use XML traditionally.

So the difference between the cloud example and this one premise, web example, is in the elements that we use in the pipeline.

So instead of the, if we look at the cloud, getting started with cloud, pipeline configuration, you would see the cloud request engine, builder, and then device detection cloud engine that I highlighted that it extracts data from the cloud request engine, but in the, In the premise pipeline, you would see only device detection hash engine.

This is the one that reads the data file from Disk, and performs device detection, basically using it.

So, and then...

So just to be clear, what was happening earlier is you've got the engine that does the request of the cloud, and then you've got another engine that turns it into that common interface format.

So all the consumption code is exactly the same.

Yes.

Yeah.

So, after that happened, it doesn't matter, like, for the downstream code, it doesn't care where it got the date.

Your models and views don't care about where the data came from.

Yep.

So let me start getting started web.

on premise.

Cool.

so let's see.

So it listens on 881.

and again, we are looking at the own premise example at my Mac.

Yep.

Let's switch to...

I see an iPhone.

Yeah.

Are we on the right port?

I hope so.

Yeah?

So we are on the own premise.

So this is on premise example.

Mm hmm.

Web integrational premise example.

And we see exactly the same properties that we solve for the cloud detection, but in this situation, it says, home and values detected using the own premise device detection data file.

Yep.

So, detection engine, using a tech data file.

And all that metadata is available.

We're not going to go through that in this particular webinar, but you can get all the data about data from these data files.

right.

So this covers IP intelligence and device detection, like, on their own.

Let's bring them together.

Yeah.

Yeah.

So we're now going to switch to pipeline, the single, again, a single pipeline that consumes both these engines in the same pipeline.

Right.

So this set of examples, they live under IP intelligence Java.

And we're just going to use on premise for this demo, we're not going to go back over the cloud again.

Right.

And they leave under, well, under, web, there is a mixed set of examples, and under console, there is also a, getting started mixed example.

So let's start with this one, probably.

So, um...

This is a command line example, so let's see how it configures the pipeline.

So, It uses the pipeline options, so it uses this getting started mixed XML pipeline configuration, and you can see that this pipeline has got 2 engines.

One is device detection engine.

Dubai detection, hash engine, has been the name of the algorithm.

We often get a question.

What is hash?

What is being hashed.

So hash is not some particular hash, not MD5 hash.

This is the name of the algorithm that is used to do device detection.

So, NIP intelligence, a premise engine.

So those are to engines that form this pipeline.

So...

And just, well, usually in switching, the hash being referred to is we use a rolling hash in order to perform the device detection on the evidence data.

So we decided that was the name to call that particular algorithm.

Sorry, for the confusion, but it stuck my mouth.

Well, it's still hashes.

Yeah, not any hash of a data file.

Yeah, a different way.

A different hash.

So, okay.

So we've got this example.

So let's try to So this is getting started on pram, this is a console example.

So we are a mixed console example, getting started.

Sorry, we should be running, getting started mixed because we are bringing together the device detection and IP intelligence.

So this is console, can insert it mixed example.

So let me try to run it.

And we get, we get, we supply mixed evidence here.

So, mixed meaning, we supply, user agent, and IP, IP address, both are hard coded for the purposes of the console example, and we get device detection results, and IP intelligence results as as output.

In the same flow data out.

Right.

It's in the same flow data.

So they are retrieved in code, in the same time, basically.

It's arbitrary, whether we want to display them in some particular order or use in other ways, like mix and match.

Because it's a command line example, you don't see the precise iPhone model detection, because we don't have a JobScript element that is running. In the browser, on page, and extracts client side evidence, so we have just a list of iPhone models here for device detection.

so I, yeah, I guess this is less interesting than the web example.

Well, it's very interesting for developers, so I can't stress this enough.

Start with the console web examples.

It means you don't have to worry about all the HTTP and the web surgery stuff and things like that.

And you can really get a feel for that.

How do you build your pipeline, configuration, or do you use the fluent builder in code, how do you then populate evidence into your flow data, then how do you process the flow data, and then finally retrieve the results?

Those are the concepts you need to understand and the console examples of the place to start, even if they're not as pretty or as flexible.

As the web example.

Let's switch over to that.

Right.

So, for that, I will, so let's look at the, so we, we look at the web, get inserted, mixed, So this is the web mixed example.

And let's look at the configuration first.

So the pipeline configuration for web is more sophisticated than for console, but the similar thing here is that it has got to engines.

It's got device detection, engine, and it's got IP intelligence on premise engine as well.

And then it is followed by the usual suspects for web, which are Jason, Builder Element, and Childscript Builder Element.

ChildScript is a client side evidence collection.

What allows you to do client site evidence collection.

so, yeah, otherwise, it's pretty similar to other, web examples.

Well, let's run it, maybe and see what it displays.

So we are gonna use this web getting started, web mixed.

And then on the command line.

I actually provided both device detection and IP intelligence data files here.

That's going to override configuration.

Right, just for the purpose of this example.

Yes.

So, let's run it and let's see what port it uses.

Maybe we'll need to switch to.

So it uses 80, 81, and we are using 80, 81 also.

So, that's good.

So let's see what we have in the let me just SO.

, you the browser so navigate here.

So this is a Lubak IP address.

You don't get much interesting data for the loop back.

So that's why we are gonna use our reverse proxy for the for our current AP address.

So, I don't quite know why.

Um.

Right, so .

I think previously we we were getting higher higher numbers somewhere here, but anyway.

Let's come on to that in a minute.

Yeah.

Right.

Well, at least we see that there is a correct device detection results on the left, then we see there are IP intelligence detection results on the right.

We can also check how this looks on the iPhone.

Let's refresh the page.

Oh, on on my phone, actually, I'm I'm running a VPN.

So, I am located differently than in reality.

So...

So you're VPNing to the US, right?

Yeah, and VPN into the US, and I've got a pretty complex region here outline for the VPN.

You can see that the region is multipolygon.

Yeah, so lots of different service areas for this VPN skewed towards the east coast of the US, but not exclusively the US.

Right.

So, but basically, I am iPhone 12 Pro Max, the same iPhone that we were detecting in the previous examples.

and there is a lot of, interesting properties, for IP intelligence, I think, do you want to highlight those?

Yeah, so what we've done here for this demo?

We've added some of the other properties that you can see on the iPhone at the moment.

So we've got the location confidence.

So we're very confident that the location area that's being returned is correct.

The way we work is look at samples from real devices, and then we train the algorithm based on those samples, so we can be pretty clear that this location is correct, but it's a very wide location area.

And that would kind of make sense because we've got a connection type of hosting or anonymous.

And we've got human probability 7.

So this is a score between one and 10. 7 makes a lot of sense here because it is a VPN.

So it may not be human.

so 7 seems quite reasonable.

And then we have the diversity numbers for hardware, browser, and platform.

So this is looking at the diversity of different models, different browser versions, and different operating systems.

And we can see these are all relatively high, and a domestic connection or a cellular domestic residential connection might be relatively low, whereas a VPN would be quite high.

So these are the numbers that you can then start to use in your solutions, because actually detecting something is definitely a VPN, is quite hard and just because it's a VPN doesn't necessarily mean it's not a human.

So how you respond to that data is going to depend on your use cases.

If you're geo-fencing content, then knowing the country's emphasis on plural that the device might be in or the user might be in, is important, if you're dealing with financial fraud or banking fraud or something like that, then you might have a different view.

So these numbers provide you that.

Now, sometimes we don't have enough samples, which is what we're seeing for the IP address on the right-hand side.

So we're coming back with 0 to integrate.

We just don't have enough information to be able to populate those diversity figures.

What we can also see there is the location confidence is medium.

So high, we would suggest, is fine for making business decisions.

Mediit rather depends on the business decision, and of course, low effectively, we're saying we just don't know enough about this for you to be able to make a business decision based on the result that you're getting back.

So those are more around the business side of things about how you use the data, but it's really important, this demo demonstrates the difference between the two.

So am I right to think that the lower the diversity number is, the more you can trust this request, sort of?

Like, it is less suspicious to you?

Because, like, this is not a VPN, this is, like, a genuine, human or genuine user.

Well, diversity is just that it's diversity.

So if you had an IP address that was being used by a crawler, for example, you would expect the diversity figures to be very low.

Right.

But then with device detection, you would expect it to come back and say, yeah, this is a crawler.

Yeah.

If it's a low device diversity and it's not a crawler, then that might indicate a IP address that's tied, particularly if the connection type is residential, that's tied to a particular location.

You might find that the geographic area then would also be quite low.

So what we're doing is providing 51Degrees users the information, we're not hiding anything from them.

How they use that data is gonna depend on their use case.

And this is one of the key problems, why we decided to reinvent IP intelligence.

We did a webinar on that a month ago.

The recording is available.

And we go into a lot more detail on that.

But basically just to come back and say something is or isn't a country or is or isn't a VPN.

It's just the way the Internet works.

These days, that's not really being honest, you're hiding something.

What we're doing is providing the data behind that.

Makes sense?

Yeah, I think this is very useful.

Yeah.

And what you've seen, just to sort of close off this section before we go into the home straight, is this data can then be combined in flow elements, in engines.

So everything you've seen today with that configuration, whether you build from, sorry, whether you build from configuration, or if you build from code using the fluent builder, you can create your own engines.

Yeah?

So, a customer might take the data from IP intelligence and device detection, everything we're seeing in this final example, and then form their own view.

So they could say, actually, we've monitored traffic for the last month, we can see what average device diversity, hardware is of people who eventually go in to login, you know, so they could actually look for their own services and say, okay, where are we getting people from?

What are the diversities associated with what we know are real customers because they go into register?

And that's going to be really different, to say, a global publisher or a holiday company or an e-commerce company or something like that.

So it's going to differ.

And then they can decide, okay, well, we want to treat certain things as suspicious based on these criteria, that could be frequency of requests, whether it's a crawler or not, age of the device, things like that divided in diversity.

So they can really build their own engines to support that.

I think you're preempting me by going to the developer documentation to explain that concept.

Right.

Right.

Yeah, so the concept about like behind the pipeline and custom flow elements is well explained in the documentation.

So if someone needs to build a custom flu element, or ancient that consumes the upstream data that comes from other from device detection, engine, or from IP intelligence engine, they definitely can use this same interface, same concepts that are described here.

This is very flexible approach, and very convenient to do in code, or, especially in configuration, for, you know, to change the pipeline without having to rebuild that. Application, basically, or modify the code of the application.

Yeah, exactly.

You can, it's extremely flexible.

And that's one of the benefits.

You know, we're not providing you a CSV file that you then have to go and do all that complexity.

So as a developer, you can be so much more productive.

If you've got any questions or issues, then of course, it's all on GitHub, so you know what to do.

Let's just look at 2 final thoughts.

So for almost all our high performance applications, people will actually use the on premise.

So cloud is great if you're a small website or you're just occasionally needing this data, maybe at the very beginning of an interaction.

The on premise is kind of where the high performance action is.

Let's look at how that data file's updated and just explain to the audience how that works.

right.

So we have actually a pretty good documentation page under the API features.

So this is a feature of the pipeline, because it applies to both IP intelligence and 51 and device detection.

And this, it is a pretty flexible data update service that is part of both engines, both IP intelligence and device detection.

So it takes certain parameters, for each of them, the sensible defaults are configured, but the main thing is that for device detection, it currently can use either our distributor service, that is documented on the distributor page, that the main parameter to which would be the license key, and basically the product type would correspond to the device detection data file.

IP intelligence at the moment uses the, yeah.

Yeah, we need just the same thing, basically.

Yeah, it does pretty much the same thing.

So, there are, So the flexibility of this data update process is that you can do it in in several different ways.

So you can either configure it to auto update and, like, for example, do it on startup or with some regular polling interval, check, whether the new data file is updated, usually you can check, like, maybe the data files are usually updated daily.

Monday to Friday, yeah. For the device detection and we're about to go to 7 days a week for IPI.

Right.

So, and then, so you can configure this polling interval to be maybe, like, every hour or every half hour to, don't miss that window when the new fresh data file just has been updated, so you just download it right away.

Or you can do it differently.

For example, if you have a cluster of machines and you don't want every machine to actually do this point on their own, you can download the data file using some script to a central location and then distribute it through the custom URL to all of your machines in that cluster, using this data update URL.

Or, for example, you can, yeah, do the 3rd way, for example, instead of doing a, instead of having them pull the data, you can actually push the data onto them and use the file system, watch your feature, that will check, it actually watches the data filing.

When it is modified on disk, it refreshes the data in memory for the engine.

And, yeah, so this is like...

So, 3 different methods.

You can either get it from us, or you can download it from us to some intermediary location, and then you can have your apps polling that location or using a file system, watcher, depending on whether you prefer HDDP or whatever your network file system protocols.

Right.

Yeah, so this is as flexible as it gets.

let us know through give how people, if it's not flexible enough for you.

Like, but you can figure it just the same way using the...

We've got an example, I think, for auto update, haven't we, for device detection?

right.

So for, yeah, so it's a different set of examples.

I think we have it for IP intelligence as well.

But it's just a pretty large data file and we are not gonna, well, let me check in the console examples.

I think it's under.

Um...

Maybe I think it was an update data file, but I don't see it right here.

No, we'll have to come back to that.

Yeah, it's right here.

We did scroll down.

Yeah, sorry.

Yeah.

So this updated I file, it doesn't use the XML pipeline configuration, it does the configuration in code, but it's dual.

I mean, you can do one or the other.

So it's the same set of methods.

So it takes the license key.

it takes, you know, enables out of the update, does it on startup, and then it has a polling interval configured.

And by the fullest 30 minutes, but like, we can supply a custom number.

For example, I don't want to update it like very often.

I just want to put some large value here and not do it at all.

Like, maybe I, in a year, I don't want it, because I, I might be doing it differently.

Well, we do advise people to keep it relatively fresh.

you know, if the new IOS version comes out, then we really want you to have the data file so that you've got the latest data, both device detection and IP intelligence changes pretty, pretty frequently, the internet's changing in terms of its network allocations, and new devices, new versions are coming along all the time.

And whilst we have good predictive capabilities, you know, if there's a major new version or a new device that comes out, you do need to get the latest data file so that you can get the latest data from your subscription that we've added to that.

Let's just finish off by talking a little bit about performance.

So we've talked about in memory and on disk, but we haven't really explained that a great deal.

So if you're updating a data file that is being consumed from disk, then you will need to have a temporary copy of that data file, if you want to automatically update it, because if you haven't got that temporary file, then we won't be able to update the primary, main version of that data file.

And the version that you're using is going to depend or sorry, how your system is operating is going to depend on the performance profile associated with the configuration or the build of the engine.

So I think we've got, if we go to console, just to finish off for IP intelligence.

You see the biggest difference just because the file sizes are a little bit larger.

We're using low memory.

Yeah, so we are reducing a low memory for, let's run the console example, maybe, it's a, it's gonna be a console getting started on prem, example. For API.

So, you see, we got the output pretty fast.

Let's switch it to next performance.

So what was happening there is it's just starting uploading the bare minimum?

Whereas this is loading everything into memory.

Yeah, and you see, like, a pause here before it actually gives...

Another couple of seconds, would you say?

Yeah, but this is a pretty fast machine.

I think our viewers are using pretty first machines, even in production.

Right.

So what happens here when you set it to next performance before starting the engine, it would read the entire 4 plus gigabyte data file into memory, to use it from memory.

So it gives it the max performance when you do the detection.

but, it consumes a lot of memory, and it also takes time to vote.

So this is the trade-off.

So, and you notice it when you actually started up the service.

Whereas when you do low memory, you know, it's gonna start pretty fast, it's not gonna consume a lot of memory.

But the performance when it does, the detection is slower because it has to go to disk every time.

So this is the trade-off.

So you've got different, there are other, there's a balanced performance option and things like that where it can load more frequently used data into memory, et cetera.

So you've got a choice.

So if you're, say, dealing in an HTTP environment, you want to do, say, 5000 requests a 2nd, then probably low or balanced is perfectly okay, because the extra few milliseconds that it takes to consult the disc per request, it doesn't really matter.

If you're doing massive batch processing or you're involved in ad tech, for example, where you might be doing 1000000s of queries a 2nd across your estate, then probably that extra few seconds, assuming you have a machine of equivalent capability, is probably worth it.

That trade-off is probably worth it.

So you can decide.

But the key thing is, you don't need any additional infrastructure.

You don't need another server, you don't need to make HTTP calls.

If you want to configure your own version of our cloud.

We have an example of to show you how to do that if you want.

But if you want to consume all this data in process, in application on premise, then you've got that option and you can decide how you go about doing that and then how you update that data, to keep your estate up to date.

Great.

Any major things we've missed?

I don't think so, but I think there is a comprehensive documentation on the website, there is GitHub.

So if we missed anything, we encourage our audience to check out those resources.

Yeah, so you know how to use GitHub?

Can raise an issue if you've got a question, of course, you can contact us. Via the website.

So that all that remains really needs for me to thank Eugene for taking us through that live code demo.

Do check out those examples, we are going to be back in a fortnight to go through go, a very different programming language, really high performance, very close to C, which a lot of the underlying algorithms are written and if you want to find out more about how to do this in go, then join us in a fortnight's time.

Thank you very much, and we look forward to seeing you soon.

Thanks for watching.