Search Engine Architecture Generator

Documents are crawled/ingested → processed (parsing, tokenization, stemming) → inverted index created → stored in search engine (Elasticsearch/Solr). Show distributed indexing, shard allocation, and replica management. Include analyzers for text processing and mapping definitions.

Indexing is the write path (documents → processing → index storage), while searching is the read path (query → index lookup → scoring → ranked results). Visualize separate pipelines with indexing optimized for throughput, searching optimized for low latency.

Use n-gram tokenizers, edge n-grams, or completion suggesters in Elasticsearch. Show user typing → prefix query → suggestion index → ranked suggestions returned in milliseconds. Include popularity scoring and typo tolerance with fuzzy matching.

Create aggregations on product attributes (brand, category, price ranges). Diagram shows search query → Elasticsearch aggregations → facet counts → UI filters → refined search. Include multi-select filters and dynamic facet updates based on search context.

Use NLP models to understand query intent and document meaning beyond keyword matching. Show vector embeddings (from BERT, Sentence Transformers) stored alongside documents, query converted to vector, nearest neighbor search in vector space. Combine with traditional search using hybrid scoring.