Learn→Embeddings & Similarity Search→Embedding Generation (BERT, Sentence-BERT, Graph Embeddings)→3 of 6
Embeddings & Similarity Search • Embedding Generation (BERT, Sentence-BERT, Graph Embeddings)Medium⏱️ ~2 min
Graph Embeddings for Collaborative and Structural Signals
Graph embeddings capture relationships and network structure that text alone cannot represent. While text embeddings encode semantic content, graph embeddings learn from connectivity patterns: which users follow which creators, which products are purchased together, which entities link in knowledge graphs. These collaborative signals are essential for recommendation systems where user item interactions define relevance more than content descriptions.
Two major approaches dominate production systems. Random walk based methods like Node2Vec generate sequences of nodes through biased random walks, then apply Word2Vec style skip gram objectives to learn embeddings where nodes appearing in similar walks cluster together. Neighborhood aggregation methods like Graph Convolutional Networks (GCNs) and GraphSAGE iteratively aggregate features from neighboring nodes, building representations that blend local structure with node attributes. Pinterest's PinSage uses GraphSAGE over billions of nodes and tens of billions of edges, generating 128 dimensional embeddings that capture both visual similarity and collaborative pin board patterns.
Dimensionality choices balance expressiveness with serving efficiency. Graph embeddings typically use 64 to 256 dimensions, lower than text embeddings, because memory constraints are severe at billion node scale. At 1 billion nodes with 128 dimensions in float32, raw storage requires 512 GB just for vectors. Product quantization or float16 compression is mandatory. Training happens offline on batch cadences because graph processing is expensive: sampling neighborhoods, aggregating features, and backpropagating through graph structure can take hours to days on distributed clusters.
Freshness versus cost is the central tradeoff. Graphs evolve as users interact, new items appear, and popularity shifts. Stale embeddings miss emerging trends and cold start new nodes. Pinterest refreshes embeddings periodically to balance freshness against the compute cost of retraining over billions of edges. Spotify balances collaborative signals from listening history with content features to handle cold start for new tracks that lack interaction data.
💡 Key Takeaways
•Graph embeddings encode collaborative filtering and network structure, capturing signals like user item co-occurrence and entity relationships that text embeddings miss
•Random walk methods like Node2Vec generate node sequences and apply skip gram, while neighborhood aggregation methods like GraphSAGE iteratively pool neighbor features over 2 to 3 hops
•Production systems use 64 to 256 dimensions for graph embeddings to manage memory at billion node scale, versus 384 to 768 for text embeddings
•Pinterest PinSage trains over billions of nodes and tens of billions of edges, generating 128 dimensional embeddings refreshed periodically, serving recommendations under 100 millisecond tail latency
•Cold start is a major failure mode: new nodes lack interaction history, requiring content based fallbacks or hybrid approaches that blend graph and text embeddings
•Training cost is high: processing billion edge graphs takes hours to days on distributed clusters, driving batch refresh cadences rather than real time updates
📌 Examples
Pinterest PinSage uses GraphSAGE over pin board graph with 128 dimensional embeddings, retrieving hundreds of recommendation candidates per request in under 100 milliseconds
Spotify combines collaborative embeddings from listening history with audio content features to handle cold start for new tracks with zero plays
Knowledge graph reasoning systems use graph embeddings to predict missing links, for example inferring that two entities likely share a relationship based on neighborhood similarity