Search Engineer

About ConsensusConsensus is an AI search engine for scientific research. We use LLMs to help millions of users find and analyze research papers, easily. Our Series A was led by USV, with major participation from top AI investors like Nat Freidman and Daniel Gross. Consensus has been featured in The Wall Street Journal, The Atlantic, The New York Times, Nature, and a16z as one of the most exciting new AI search engines in the world.Our mission is to make expert knowledge discoverable and consumable for all. Help us build the future of research.ResponsibilitiesImprove search relevance and query understanding via ranking models, feature engineering, and ML model integrationBuild dashboards and tools for evaluating search relevance and engagement metricsManage search infrastructure, optimize latency, and reduce costsDevelop search data pipelines for downstream relevance and ML tasksContribute to general full stack web development supporting search, user-facing features, and other team needsMust Haves5+ years of software engineering experience3+ years of search relevance and NLP experience, including experience with semantic search, learning-to-rank, and modern NLP techniquesStrong experience with PythonNice to HavesStrong experience with ElasticSearchNLP and ML libraries, like XGBoost, NLTK, spaCy, and scikit-learnDistributed data processing frameworks like Spark, Dataflow, Flink, or DaskData analysis and visualization libraries, such as pandas, numpy, scikit-learn, seaborn, plotly, etc.Knowledge GraphsWeb CrawlingExperience constructing domain-specific golden sets, especially in science and medicine domainsOther generalist full stack engineering skillsWhy You'll Succeed:Attention to detail and craft.Track record of high velocity delivery.Interest in science, research, and LLMs.

Created: 2025-01-24