Відео

hello brother, i need your help in something, can i get your ig or something to chat?

bro you dont even know what you are teaching....you are reading all the stuff from another screen,....

Appreciate the energy of the guy :)

how to do with mongodb ?

Brother i have been able to install pyttsx3 on my M1 macbook air, all u need to do is to add 'pip install py3-tts' since this version contains pyobjc 9.0.1version. I am feeling like a dumb commenting on a 5 years old video like this😅, but it might help someone else..

I have a csv file and when i am using concat function it automatically name unnamed group 1,2,3... Also the alignment gets messy with songle line of code How to fix it

Pass parameters from c# to python and parse the result to c# class object

Thank you for update, keep going!

Git hun link please?

Greetings Mr Soumil. I want to congratulate for this interesting video. I didn't know this HUDIs streaming capacity and it's awsome. I need to said about your example, It's a little confusing because for most common DE porceses we part form a log to a concrete table I mean there's an agregation. I mean I was expecting the totally reverse ( that new fields was inserted in the origin hudi table log like and then to agregate to the dimensional table to take just one register of each customer ). I need to read more about HUDI streaming. You Rock!!

Please help me to implement this project.

thank you so much, this is very helpful. keep doing what youre doing 🫡 hoping to see more long videos and comprehensive projects

Can you help me with something?

have you done oracle cdc through apache flink

Hi Soumil, thanks a lot for this amazing content. I am starting in the world of data streaming and it’s a really useful case! I am facing an error accessing your code in GitHub. Do you know what can be te cause of it? Best

CDC part is missing. please add

route 53 failover with primary and secondary resources is active-passive, not active-active

In multithreading, creating multiple threads increases the chance of using more CPU cores (assuming each task takes 1 millisecond). However, Python's Global Interpreter Lock (GIL) prevents true parallelism for CPU-bound tasks within a single process. Only one thread can execute Python bytecode at a time. For I/O-bound tasks, other threads can utilize other CPU and execute bytecode meanwhile, creating the illusion of concurrency. However, managing multiple threads can be resource-intensive due to memory overhead. In contrast, async utilizes a single thread with an event loop. You submit tasks to the event loop, and it executes them sequentially. When a task encounters an I/O operation, the event loop tracks its progress. The main thread then executes other tasks in the queue, if any. Once an I/O task completes, the main thread continues executing the remaining CPU-bound part of the task.

Appreciated

Very interesting. Just curious on how can we manage the costs of cloudwatch. since cloudwatch is one of those services that at the beginning is quite cheap. but with time (thousands of tables in my case) those costs add up to quite a lot

helpfull video soumil! But is still not really clear what the relation is between all of the 3 configuration. the max is understood. but what about the min and commits.retained? ATM I am only using commits retained and I'm exploring these configuration if I need to add them or not

Love your videos Soumil❤

thanks a lott mann

Really helpful for those who want to learn airflow.

@Soumil :- your voice is quite clear which software you used to record and show these end to end

did this nibba just pronounced chile as cha-aisle

How to do interpolation for categorical variable

Hi, this setup spike my billing very high, The setup was to build lambda function to read the latest file from the s3 dir and make transformation then finally to s3 target dir, So this all setup with the python script has to run once the s3 notification to lambda function that an file just came to s3. But it went into a loop and made the s3 and lambda billing spike Let me knew what is the issue in my setup that i didn't noticed at first while running this python script in lambda

This is such a great video I have ever found! I need do a log tracker for my team with kinesis/firehose/S3/Anthena as you did, but I have a question is that, can we connect S3 with Anthna directly? Or say, we have to need AWS glue to do so ?

be more delicate with your keyboard

This is amazing. I learned a lot. I want to come to India to study Data Analytics

You want to use alias record for the route 53 as it’s free for any number of invocations for any aws resource. Instead of typical A record.

How the duplication is handled ? If I have a 2 source tables order and orderDetails - then when changes are done in both tables - how to combine the data and save ? Or it des not required to combine - only in the Athena level or glue job we can combine the 2 tables and wrote a single records to s3 ? Can you please explain. soe where I read - from kafka topic - a kafka stream consumer can be used to perform the join operation before send it to another topic

Amazing FREE content, but when covering the code, we don't need to see your wonderful face. We need to see the code.

How to find out the top 10 records?

After 20 minutes - I get: Error Category: UNCLASSIFIED_ERROR; An error occurred while calling o98.purgeS3Path. Unable to execute HTTP request: The target server failed to respond. I have been trying to figure it out but it happens everytime after 22 minutes. I was able to run the job earlier and it worked on 2 million objects.

शानदार👌

Try using the cloud itseld instead of the local elastic

How to make the query dynamically passed instead of the one query and how to make the fitler inside the query dynamic also

where is the link ?

Heeey Soumil! Been following you are a while now and think you channel is great! I am using dbt with a thriftserver, on my local server, trying to push data to Azure adls gen2 as delta tables. But struggling to get it to work. Would be awesome if you could do a video about this!

Very Informative. QQ- is this table to be re-created every time underlying new S3 files are added in data lake or it captures new data automatically?

tnxxx buddy :)

Lol did he just clicked through all. I don't mind if you take some time to explain