Designing the DB Watermark Polling using Scheduler in Mule 4
Вставка
- Опубліковано 30 вер 2024
- This video explains the design technique to read the new records from the Database and process them as it comes. The design is to use the Object Store value as a watermark and schedule the process using Scheduler.
ObjectStores's Retrieve(userid) & Store (id) objects should point to the same key name....either both should be 'userid' or both should be 'id'.
Hi i have also got this doubt. Both should point to same id right ? Thanks in Advance
Yes correct
Hi Senthil - If the DB table structure has the metadata to have auto incremented ID, the value will always be greater than the previously available IDs. If not, we can design the watermark using the field with the timestamp and set the timestamp to now() back into the object store key so that the recently-inserted records are fetched always. Also, you can use the combination of field values as a key and split it using flow variables and use them in the criteria. Cheers !!
how can we use timestamp for lastmodified record should be populated... so in the object store i set now() then what should i give in objectstore retrieve ???
Hi Ravi - Yes, you are right. Please use now() as a last step and format it as per the DB you are using. Before setting you need to play around with the query to check if the timestamp format is correct and then place it in the object store value. Cheers !!
@@sivathankamanee-channel not clear Siva 😔
Hi Siva
Your explanation is so good. Thanks for your effort...
can you please clarify how it is retrieving key from object store if we store it in key - id and retrieving using key - userId
Sir how can we retry for if the database connection is down for sometime we need to retry for every one hour how can we design this scenario
I want to take data from salesforce after every 30 minutes but data from salesforce should be fetched based on last run time of the mule flow
Nice video Sir,
Can you tell if the poll/scheduler stops sue to any reason.... can we use a manual api implementation? If yes, can you make a video please ?
Can you plz explain batch job failure records logged in separate file while inserting the duplicate records.
In the above example, different key names used in the Retrieve and Store. Can you please elaborate how it would consider different keys having same data?
at Retrieve - userid has mentioned as a key
at Store - id has mentioned as a key.
Hi Siva, could you please do a video on if the error occurred in a flow how we can store the errored details in the database I am thinking about the Scenario could you please suggest me
Can you explain about any mule projects that will be useful for more students regarding interviews please try sir
Can this be down when retrieving a CSV file hourly to exclude the data that was returned with the previous hour?
is this a correct scenario or not? please correct me ...
insert record with id value as 3,it wont get the record, since select stmt is retriveing id >13 will only retrive, how to void it
siva can you upload configuration XML for this?
Hi Siva,
Can you create a video on threadpools please..
Hi Siva,
you videos are amzing for learners. Thank you for ur efforts.
i am unable to do below 2 schenarios can u do it for us?
Scenarioes
1)you are loading 100 records to the database and 50th record failed due
to some reason. non of the record written in database. how?
2)100 records are processing through batch and 20th record failed. you dont want to
stop the batch processing and raise exception for that 1 record and continue the batch
processing .How you will raise exception?
(or In batch process if there are 100 records and 20th record got failed. But I want
process all the records and rollback only 20 record...how?)
Hi Subba - Will respond shortly. Thanks.
Really useful. Keep up the good work