I like most of this but it feels like it could be even more open-ended. Perhaps this could be further improved by randomizing the target dataset? Instead of taking a fixed dataset as target, take a random subset from a massive combined one. Obviously, by default, this will be very noisy. However, perhaps the way images are drawn from the dataset could *also* be learned: If two images tend to do well on the same search queries and poorly on the same search queries, make them more likely to be drawn for a dataset together, and if they tend to oppose each other, make them less likely to be drawn together. Basically, draw entire distributions of images, which you can then refine over time. A simpler method might be to just rely on CLIP, take a random direction in CLIP's embedding space, and simply sample based on *that.* - Then it tries to learn how to map CLIP vectors to search queries.
thanks for sharing!!
Interesting work, thanks for sharing.
I like most of this but it feels like it could be even more open-ended.
Perhaps this could be further improved by randomizing the target dataset?
Instead of taking a fixed dataset as target, take a random subset from a massive combined one.
Obviously, by default, this will be very noisy.
However, perhaps the way images are drawn from the dataset could *also* be learned: If two images tend to do well on the same search queries and poorly on the same search queries, make them more likely to be drawn for a dataset together, and if they tend to oppose each other, make them less likely to be drawn together.
Basically, draw entire distributions of images, which you can then refine over time.
A simpler method might be to just rely on CLIP, take a random direction in CLIP's embedding space, and simply sample based on *that.* - Then it tries to learn how to map CLIP vectors to search queries.