When you first start building a search engine in Rust, your instinct is usually to be as efficient as possible. You don't want to copy strings over and over again, so you reach for &str. It seems logical: why store a whole String in your index when you can just point to a slice of the original text?

I remember doing this exact thing in one of my first indexing projects. I wrote this code, thinking I was being clever with memory:

use std::collections::HashMap;

struct SimpleIndex<'a> {
    // Map a word to a list of document IDs
    map: HashMap<&'a str, Vec>,
}

impl<'a> SimpleIndex<'a> {
    fn add_document(&mut self, id: usize, text: &'a str) {
        for word in text.split_whitespace() {
            self.map.entry(word).or_default().push(id);
        }
    }
}

fn main() {
    let mut engine = SimpleIndex { map: HashMap::new() };
    
    // This looks fine...
    let doc = String::from("Rust is fast and safe");
    engine.add_document(1, &doc); 
    
    // But wait, if we do this in a loop with dynamically loaded files...
    // we run into a massive lifetime headache.
}

The Lifetime Trap with String Slices

The code above compiles if doc lives as long as engine, but in a real search engine, you're loading files from a disk. If you load a file into a local String variable, process it, and then try to store references to that string in your HashMap, the borrow checker will stop you dead in your tracks. The String is dropped at the end of the loop iteration, leaving your index with "dangling pointers."

You'll see an error telling you that &doc does not live long enough. You might try to fight it with 'static or complex lifetime annotations, but that's a rabbit hole you don't want to go down for this specific problem. In a search engine, the index is the "source of truth"—it needs to own the data it is indexing.

Owning Your Keys with String

The fix is simple but feels "expensive" to a beginner: use String as the key in your HashMap. By converting the &str from the tokenizer into an owned String, the index becomes independent of the original document's memory lifecycle. Yes, there's an allocation, but for a search engine, the stability of the index is worth the cost.

use std::collections::HashMap;

struct SearchEngine {
    // We use String here so the engine owns the words it indexes
    index: HashMap<String, Vec<usize>>,
}

impl SearchEngine {
    fn new() -> Self {
        Self { index: HashMap::new() }
    }

    fn index_document(&mut self, id: usize, text: &str) {
        for word in text.split_whitespace() {
            // Normalize to lowercase so "Rust" and "rust" are the same term
            let normalized = word.to_lowercase();
            
            // The entry API is your best friend here. 
            // It handles the "if key exists, push; else, create vec" logic in one go.
            self.index.entry(normalized).or_insert_with(Vec::new).push(id);
        }
    }
}

Refining the Search Query

Now that we can actually store data without the borrow checker screaming at us, we need to retrieve it. The most basic search is a "term lookup." But since we stored our IDs in a Vec, we have to decide if we want to return a reference to that vector or a clone of it. Usually, returning a reference is better for performance.

I prefer using Option<&[usize]> for the return type. Using a slice [usize] instead of a Vec is a professional touch—it tells the caller "you can read this list, but you can't modify the internal index."

impl SearchEngine {
    fn search(&self, query: &str) -> Option<&[usize]> {
        let normalized = query.to_lowercase();
        // We return a slice of the vector stored in the map
        self.index.get(&normalized).map(|v| v.as_slice())
    }
}

fn main() {
    let mut engine = SearchEngine::new();
    engine.index_document(0, "The quick brown fox");
    engine.index_document(1, "Jumped over the lazy dog");
    engine.index_document(2, "The fox is quick");

    if let Some(docs) = engine.search("fox") {
        println!("Found 'fox' in documents: {:?}", docs); // [0, 2]
    }
}

One last thing: in a production system, you wouldn't just use split_whitespace(). You'd use a regex or a specialized crate to strip punctuation. For now, we're keeping it simple, but keep in mind that "fox," and "fox" are currently treated as different words. In the real world, that's a bug; here, it's a simplification.

📋 Practical Task

Implementing a Boolean 'AND' Search Filter

Currently, your SearchEngine can only search for one word at a time. Your task is to implement a new method called search_all that takes a string containing multiple words (separated by spaces) and returns only the document IDs that contain all of those words.

Requirements:

• The method signature should be: fn search_all(&self, query: &str) -> Vec<usize>.
• You must handle the case where the query is empty (return an empty vector).
• If any one of the words in the query is not found in the index, the entire search should return an empty vector (since no document contains all the terms).
• You will need to find the intersection of the document ID lists for each word in the query.

Hint: Consider starting with the list of IDs from the first word and then filtering that list by checking if each ID also exists in the lists for the subsequent words.