Name: Ask On Data
Uploaded: 2024-12-09T09:04:11.630Z
Duration: 2 min 33 s
Description: An open-source, AI-powered chat-based ETL tool that simplifies data engineering tasks through natural language interactions.

ask-on-data

An open-source, AI-powered chat-based ETL tool that simplifies data engineering tasks through natural language interactions.

Ask On Data

Welcome to Ask On Data, a chat based AI power data engineering tool. By a very simple chat interface and no technical knowledge, Ask On Data can be used for various kind of data engineering related operations like data migration, data loading, data integration, data cleaning, data analysis, data wrangling, and data transformation. 

In this quick overview, we will show you some of the use cases and create a feline which can then be orchestrated with various configuration options like initial load, trun and load, CDC, etc. Logging into the Ask On Data application at the data sources page, you have the option of connecting to the data source of your choice, which could be database, flat files, S3 data, L platforms, data warehouses, etc. 

We continue to add more and more support like API, JSON, etc. You can reach out to us for any database support. Also, these data sources which are connected over here, they can act as source. They can access. 

Please share the AI Agent Store with friends.

Target for this specific demo we will use flat files. We have already uploaded four files which are the four sales data of a baking company. Now click and create a new job. Specify which data source will be used for this specific job. 

Job in this demo example we would be connecting to the four files which we had uploaded earlier. Once the files are uploaded we have an option to preview which can allow us to preview the data as well as understand how our instructions are affecting the sample data set. If you connect to a database similar to the files the tables of the database along with the sample preview of the data will appear over here. 

Now we will cover some examples of data integration using Ask On data. As you can see we have four years of sales data which we may want to merge into a single table. In order to do that we can simply use an Union command and over here we can type something like Union data 2017 data. 

Share the AI Agent Store with friends.

Of 2018 date of 2019 and datea of 2020 and enter once the union happens it will create a separate file as you can see here which is an union of all the four files and you can see the amount of data has also increased. You can if you would like you can even rename this specific uh file also our table also it is always advisable to give it a name which is not a reserved keyw. In a similar way you can do other kind of data integration as well well like joins. 

Now we will cover some examples of data cleaning use cases using ascon data. You can see that there are certain kind of dup duplicate records. So first of all we can remove the duplicate records by simply typing remove duplicate records from order Ting. You can see the duplicate records have been removed and the number of records have reduced. In certain cases uh we may have data type mismatch so we can also uh fix that by simply typing cast so in this case we may.

Want to cast order ID from object into string so we can uh do the same. You can see the data type has been changed from order add to string. Third thing which we may want to do is we can see here that there is some uh are not having audio ID because for them promotion campaigns were run. 

And we may want to remove these specific uh data set from our analysis. So we can uh type something like filter all records from order ID whose value is not XX XX XX. So you can see those specific records have been removed. If we want to remove this kind of Nan or n records we can simply in a similar way we can type. 

Now we will cover some examples of data wrangling using Ask On data. We have customer ID which is a combination of customer ID as well as the customer name. So we may want to split it. So we can type split customer ID using hyph into client uncore name and client into client uncore ID and centore name. Two new columns have been. 

Share the AI Agent Store with friends.

Added client ID and client name. Uh if you would like you can cast it to integer. Now customer ID is not required so we may want to drop it. 

So we can simply type drop customer ID column. Now we will cover some use cases of data transformation. So uh we have revenue and cost and we may want to add a new column which is profit which is a difference between revenue and cost. 

So we can type calculate a new column called profit which is revenue minus cost. We may want to reorder and put this profit column next to cost. So we can uh simply type rearrange profit column as the fifth column. 

The profit column has come as a sub column. We may want to find out the lag between the orders and shipment. We can type a command like new column delay shipment which is ship date minus order date. 

So we now have that specific information in nanoseconds. Now we will cover some examples of data analysis using Ask On data. In many cases we may want to share AI Agent Store with friends.

Want to to create calcul table like hourly sales daily sales weekly sales Etc and load that data into the target data lake or data warehouse on an incremental basis. In those cases these kind of queries are very helpful. First of all we will create a date wise profit report. So it has created a separate table our file which is a date wise profit report. Right now the data is not in is not sorted. So if we' like we can sort it sort order date in ascending order. 

Now this data is in send order covering another example. We will go back to the original file by typing a command like use filter value as the working file. We may want to analyze this data and we may want to understand how much percentage of orders have been shipped cancelled rescheduled Etc. So we can type a command like analyze this data and give me percentage of orders according to order status. 

So now we are having order status wise percentage of order and we.

Can see most of the orders have been shipped. There are various other operations which are available. Uh, like I, which is a job history, which over which we can see what all steps have been performed in that pipeline, which is created with details about every step. 

We even have options to undo and redo certain kind of transformation that we are interested in. For more technical control, here we have an option of YAML. So you can click on the YAML and you can see various steps which have been performed. 

And if you'd like, you can even edit those steps, edit those SQL before you run them and orchestrate them. Here we have an option of scheduling. While we have scheduling, we give option to write a full Python code over here if required. 

Over here you can specify, you can give a schedule name, you can specify which file you would like to schedule and what would be the destination, whether it would be a local storage or it. 

Share the AI Agent Store with friends.

Would be a Target database those things you can specify and then you can schedule it. 

And over here you can specify the uh frequency of sched. Apart from that there are more admin related options which are available over here which uh by which you can see how many jobs have been scheduled. 

What is their current training status error and other kind of uh information. 

Please reach out to us on support atate askon dat.com for a personalized demo and PC. 

Share the AI Agent Store with friends.

Note, that there may be slight differences in text and video. Watch video for the original content.

Ask On Data Video