@Mark, This is awesome especially node disconnection improvements. It really simplifies to alot. Believe me, we have had production issues alot for 3-nodes clusters. Occasionally, we face node disconnection issues, whenever we do restarts due to infrastructure upgrades or any outages. We had to either delete the flow.xml.gz or manually copy from cluster. This bullet proofs the entire maintenance model for NiFi. Thank you for highlighting all the features like always. Fork/Join Enrichment, Failure with Retry etc will simplify alot for developers in terms of not using common RetryFlowFile procesor template. Great improvements. Love the community. Thanks again..
That node reconnection is pretty huge. From time to time when writing a scripted processor and something goofs, a node may disconnect. No longer having to stop everything, copy over the flow from another node, start it back up, and cross your fingers is going to be great.
Hi, Thank you for your great videos. Do you know if there is any good operator for Nifi in Kubernetes? Nifi is really great but I have some difficulties ruining it in K8s. thanks
I only have a pretty beginner-level understanding of Kubernetes. Some general advice: you'll definitely want to use a stateful set. You'll want a separate volume each for content, flowfile, provenance repositories. In order to scale down, you'll want to run "bin/nifi.sh decommission" - this will disconnect the node, offload the data to other nodes, remove node from cluster, and shutdown. From the Cloudera perspective, we've put together a very powerful offering in Cloudera DataFlow Service (www.cloudera.com/products/cdf.html) that makes it very easy to deploy flows as separate nifi clusters running on k8s with auto scaling. I'm not the one to speak to how all of the Kubernetes magic happens, though.
Hello good night, could you share the code of your nifi cluster infrastructure? I'm trying to create a nifi cluster on kubernetes in version 1.16 but without success in the version. Only in version 1.13
@Mark, This is awesome especially node disconnection improvements. It really simplifies to alot. Believe me, we have had production issues alot for 3-nodes clusters. Occasionally, we face node disconnection issues, whenever we do restarts due to infrastructure upgrades or any outages. We had to either delete the flow.xml.gz or manually copy from cluster. This bullet proofs the entire maintenance model for NiFi. Thank you for highlighting all the features like always. Fork/Join Enrichment, Failure with Retry etc will simplify alot for developers in terms of not using common RetryFlowFile procesor template. Great improvements. Love the community. Thanks again..
These are great additions and changes, thanx Mark and team!
node disconnect/reconnect improvements awesome !! Thank You Mark.
Thanks Mark again dir providing These cool News Features.👍
I’m not the one providing all the features, just the one talking about them :)
Another great video Mark.
That node reconnection is pretty huge. From time to time when writing a scripted processor and something goofs, a node may disconnect. No longer having to stop everything, copy over the flow from another node, start it back up, and cross your fingers is going to be great.
Thank you Mark!
Hi, Thank you for your great videos. Do you know if there is any good operator for Nifi in Kubernetes? Nifi is really great but I have some difficulties ruining it in K8s.
thanks
I know there is one that was produced by Orange, I believe. I’ve not used it though.
@@nifinotes5127 Thank you.
Awesome
Can u please do a video on clustering the nifi on kubernetes
I only have a pretty beginner-level understanding of Kubernetes. Some general advice: you'll definitely want to use a stateful set. You'll want a separate volume each for content, flowfile, provenance repositories. In order to scale down, you'll want to run "bin/nifi.sh decommission" - this will disconnect the node, offload the data to other nodes, remove node from cluster, and shutdown. From the Cloudera perspective, we've put together a very powerful offering in Cloudera DataFlow Service (www.cloudera.com/products/cdf.html) that makes it very easy to deploy flows as separate nifi clusters running on k8s with auto scaling. I'm not the one to speak to how all of the Kubernetes magic happens, though.
Hi sir, can i set specific state for listFile. I cannot find nifi rest api for that
Hi Duc, no, you cannot manually set state for components, with the exception of clearing the state.
Hello good night, could you share the code of your nifi cluster infrastructure?
I'm trying to create a nifi cluster on kubernetes in version 1.16 but without success in the version.
Only in version 1.13
I don’t run on Kubernetes. But would recommend asking in the NiFi slack channel. Lots of people there who do, who can help.