I like the use case and I think it's missing an important point. Sure it's faster to bootstrap a GenAI model to classify personas in the beginning especially when you have no or little unlabeled data, but surely in time after collecting e-commerce data a trad supervised learning model (perhaps even with GenAI extracted features) will outperform GenAI. Would be interesting to see how this could be used with in a human in the loop system to review GenAI classifications or even use sales performance to label a data set.
Agree with you, Anything to deal with the LLM is the costliest option ever. Although one could easily do with the LLM, however I don't think it's the best option either.
Hi, thanks for the comment. A few thoughts: When you say, 'A traditional supervised ML model will outperform GenAI,' what does 'outperform' refer to in this context? Predictive accuracy? It's important to note that there are no persona labels available, nor will there be in the future. Predicting personas is not the primary business objective. The goal is to engineer tailored messaging that boosts conversion rates and value. Achieving this requires experimentally learning the function that maps features and messages to desired outcomes. The LLM approach will help speed up this experimentation process. If there are pre-existing messages or campaigns, their outcomes can be modeled using supervised learning, while carefully controlling for biases through A/B testing or causal ML. Additionally, with LLMs, there's the option to fine-tune them using Reinforcement Learning from Human Feedback (RLHF), which could potentially enhance the model's ability to generate messages that more effectively convert customers over time.
LLMs are likely already in use for surveillance purposes in some places. In this particular use case, however, email marketing with Klaviyo is done with full marketing consent that can be redacted at any time.
Interesting idea but I don't think a company would ever use LLMs to determine personas due to the lack of explainability. It would be an absolute nightmare to explain to leadership why 5% of your high value customers shifted after the last model update.
LLMs can explain why a customer falls into a given category much better than traditional ML. You put business rules on top of the persona classification to make sure you're changing the "effective persona" and the messaging at the right time. No different than the challenges you face when personalizing without LLMs.
I like the use case and I think it's missing an important point. Sure it's faster to bootstrap a GenAI model to classify personas in the beginning especially when you have no or little unlabeled data, but surely in time after collecting e-commerce data a trad supervised learning model (perhaps even with GenAI extracted features) will outperform GenAI. Would be interesting to see how this could be used with in a human in the loop system to review GenAI classifications or even use sales performance to label a data set.
Agree with you, Anything to deal with the LLM is the costliest option ever. Although one could easily do with the LLM, however I don't think it's the best option either.
Hi, thanks for the comment. A few thoughts: When you say, 'A traditional supervised ML model will outperform GenAI,' what does 'outperform' refer to in this context? Predictive accuracy? It's important to note that there are no persona labels available, nor will there be in the future. Predicting personas is not the primary business objective. The goal is to engineer tailored messaging that boosts conversion rates and value. Achieving this requires experimentally learning the function that maps features and messages to desired outcomes. The LLM approach will help speed up this experimentation process. If there are pre-existing messages or campaigns, their outcomes can be modeled using supervised learning, while carefully controlling for biases through A/B testing or causal ML. Additionally, with LLMs, there's the option to fine-tune them using Reinforcement Learning from Human Feedback (RLHF), which could potentially enhance the model's ability to generate messages that more effectively convert customers over time.
IMO GNN/GCN suits better for many of the things done here. NLP LLMs are very good as a Semantic layer and will probably stay so?
Super informative video.
And is this what 21st century surveillance looks like ?
LLMs are likely already in use for surveillance purposes in some places. In this particular use case, however, email marketing with Klaviyo is done with full marketing consent that can be redacted at any time.
Interesting idea but I don't think a company would ever use LLMs to determine personas due to the lack of explainability. It would be an absolute nightmare to explain to leadership why 5% of your high value customers shifted after the last model update.
LLMs can explain why a customer falls into a given category much better than traditional ML. You put business rules on top of the persona classification to make sure you're changing the "effective persona" and the messaging at the right time. No different than the challenges you face when personalizing without LLMs.
Wrong lol. You obviously don’t know infra ML. Too slow.