+ ihRt^RIt^RIt^RIt^RIt^RIt^RIt^RIt^RIt ^RI H t ^RI t ^RI Ht]!4^RIHt^RIHt^RIHt^RIt^RIHt]!R4]P2P5]P2P7]44t]P2P5]4t]P2P?^]4^R I H!t!]!R R R R 7t"^RI#t#^RI$t$^RI%t%R]PLR&R]PLR&]$PN!R](R7]%PR!R4PU]%PV4Rt,]!R4]!R],R7t-]!R4]!R4/t..s/Rt0^RI1t1]1Pd!]0RR7Pg4Rt4]1Pd!]4RR7Pg4RR lt5R!R"lt6!R#R$]4t7R%t8R&R'lt9R(R)lt:R*R+lt;]"PyR,4R-R.l4t=R#)/a Backend API for Dr. Robert Young's semantic search Q&A system This module provides a FastAPI application that: 1. Performs semantic search on scraped blog articles 2. Generates contextual answers using local LLM 3. Provides performance timing information N)deque) load_dotenv)FastAPI)StreamingResponse) BaseModel)SentenceTransformerz&[INFO] Using only Ollama (local mode).)get_connectionz$Dr. Robert Young Semantic Search APIzBSemantic search and Q&A system for Dr. Robert Young's blog contentz1.0.0)title descriptionversionfalseTOKENIZERS_PARALLELISM1HF_HUB_DISABLE_TELEMETRYignore)categorysentence_transformerscpuz5[MODEL] Loading embedding model (from local cache)...zall-MiniLM-L6-v2)devicez[MODEL] Embedding model ready!z[LLM] Using Ollama (Local mode)c n\R4\4pVPRR7pVPR4VP 4pVP 4VP 4.pVFlp\ P!\P!VR,44pVPRVR,RVR,RVR,R VR ,RV/4Kn Vs \R \\4 R 24R # \dKi;i \dp\R T 24R p?R #R p?ii;i)z:Load all article embeddings from DB into memory at startupz2 [CACHE] Loading article embeddings into memory...T dictionaryzDSELECT id, title, url, content, embedding FROM dr_young_all_articles embeddingidr urlcontentz[CACHE] Loaded z articles into memory!z[CACHE] Failed to load cache: N)printrcursorexecutefetchallclosenparrayast literal_evalappend Exception article_cachelen)conncurrows new_cacherembes KC:\Users\Aakash\Desktop\Scrapping\Chatbot-Dr.Young O Robert\backend\main.pyload_article_cacher1Ms4 CDkkTk* Z[||~   A hhs//+?@  !D'QwZ1U8q| ""  M 233IJK   4 .qc2334s=A/D2A(C>"D> D  D D  D D4D//D4T)targetdaemonc ,\R4\P!RRRRRRR/^ 23rIc4a]tRt^toRtRtV3RltRtVtR#) ChatRequestz Request model for chat endpoint Attributes: question (str): The user's question to be answered conversation_id (str): Optional conversation identifier to maintain context defaultc2<V^8dQh/S[;R&S[;R&#)rCrOrDrE)rG __classdict__s"r0rHChatRequest.__annotate__sM$rIN) __name__ __module__ __qualname____firstlineno____doc__rD__annotate_func____static_attributes____classdictcell__)r[s@r0rXrXs%OrIrXc\P!W4\PPV4\PPV4,, #)z Calculate cosine similarity between two vectors Args: a (numpy.ndarray): First vector b (numpy.ndarray): Second vector Returns: float: Cosine similarity score between 0 and 1 )r!dotlinalgnorm)abs&&r0cosinerls6 66!<299>>!,ryy~~a/@@ AArIc0V^8dQhR\R\/#)rCtextreturnrE)rGs"r0rHrHsrIc \P!RRV4p\P!RRV4p\P!RRV\P\P,R7pVP RR4pVP 4#) z Clean and preprocess context text for LLM consumption This function removes unwanted formatting elements that might confuse the LLM. Args: text (str): Raw text content to be cleaned Returns: str: Cleaned text ready for LLM processing z\n?\s*\d+[\.]\)\s* u [•\-–▪]z\n\s*References?\s*:\s*$)flagsZeolitéZeolite)resub MULTILINE IGNORECASEreplacestrip)rns&r0 clean_contextr|sn 66'd 3D 66"C .D 66-r4r||bmm?[ \D << I .D ::<rIc$V^8dQhR\/#rCr8rE)rGs"r0rHrHsS<Sr?hasattrrnr iter_linesr;loadsdecoderzgetJSONDecodeErrorr&rF exceptionsConnectionErrorTimeouttyper^)r8r- error_textreceived_responselinedatachunkr/s& r0call_llama2_streamrsA"B< ]] 1*&$!3S$c!3    }}#'.q&'9'9QVV?Q .q}}oR |LM.>,0) $Z 0 8 8Y O# xx''('4%llo  \++ -c!fX66c   r    . .kjj    & &rqq < $T!W%5%5$6bQABd1g../s3q6(;;;rr;rrrzrr&rF)r8r-rrrr/s& r0call_llama2_stream_directr/s &) ]] 1*&$!3S$c!3     zz$++g"67% ,44ZKEK88F##'%    D )Ax((()sXD2CA;C6C:CD C CDC C>#C93D9C>>Dz/chatc$V^8dQhR\/#)rCq)rX)rGs"r0rHrHjsww+wrIc 4aa&a'a(a)a*a+a,a-a.a/a0a1a2a3a4"\P!4o3\SP4p\RSP 24\R\ V4 R24\P!4p\ P SPRRR7p\P!4V, o+\P!4p\4o&S&PRR7o)\P!4V, o*.p\P!4pRp\EFp\W8R ,4p 0R kR kR kR kRkRkRkRkRkRkRkRkRkRkRkRkRkRkRkRkRkRkR kR!kR"kR#kR$kR%kR&kR'kR(kR)kR*kR+kR,kR-kR.kR/kR0kR1kR2kp SPP4P4U u.uFqV 9gK \ V 4^8gKV NK! p p V 'dVR3,P4o4VPR44;'gR5R6,P4o'\V43R7lV 44p \V'3R8lV 44pV \!V R9,R:4, p V \!VR;,R<4, p W8dT pV R=8gEKzVP#W34EK \%VR>RR?7p\'4p.pVFhwppVR3,P)4P4pVV9d%VP+V4VP#VV34\ V4^8XgKhM Tp\P!4V, o2\R@VRA 24V'd=\-V^4F+wpwr\RBV RCVR3,RD, REV RA 24K- M\RFSP RG24V'Eg|SPP4o0V'g\RHSP RG24RIo,EM$V'EdV'd VRu,M/pV'd!VPRJR54P4MR5pV'd!VPRKR54P4MR5pV RLV 2p\'VP44p\'S0P44p\ VP/V44p\ VP1V44pV^8d VV, M^pVRM8gk\2;QJd4V03RNlVP4RO,4F 'gK RM) RPM%!V03RNlVP4RO,44'd\RQVRR RSSP RG24\RTVRD, RUVRD, RV24VRW, RXSP RY2o-\P!4o.V-V.V3RZlp\5V!4R[R\7#\R]VRR RSSP RG24\R^VRD, RUVRD, RV24RIo,V,3R_lp\5V!4R[R\7#\P!4p.p .o1VFwp!pS1P#R3VR3,R`VR`,/4S)P7RaVRb,34S)P94p"V"'gK]\;V"R4,4p#V P#V#4K \RcRv 24\Rd\ V 4 Re24\-V^4FDwpwr\RfV RgVR3, 24\RhVR`, 24\RiV RA 24KF \Rv 4RjP=V 4p$\P!4V, o(\Rk\ V$4 Rl\ V$Rm,4 24\RnSP 24\RcRw 24\Ro4\Rw 4\V$Rm,4\Rw 4\Rp4\Rv 4RqV$Rm, RrSP Rs2o/\P!4o.V&V(V)V*V+V.V/VV1V2V33 Rtlp%\5V%!4R[R\7#uup i5i)xa Main chat endpoint that processes user questions with session-based memory This endpoint performs semantic search on the blog database, maintains conversation context, and generates contextual answers using a local LLM. Args: q (ChatRequest): The user's question request with optional conversation ID Returns: StreamingResponse: Streaming response containing answer and references z[SESSION] CONVERSATION: z[HISTORY] LENGTH: z interactionsTr)convert_to_numpyrrgrwhathowwhyistherjandoesdocanfromtoofinandorforthisthatitimemygiveexplainexampleabouttellmeanreallyhelpneedusingafterlongtermthesethosethemtheirsuchr rrr:Ni Nc38<"TFqS9gK ^xK R#5iNr]).0w title_lowers& r0 chat..sP+;aK?O+; c38<"TFqS9gK ^xK R#5irr])rr content_lowers& r0rrs!T-=mAS!!-=rg?g333333?g{Gz?rg?cV^,#)r])xs&r0chat..s!A$rI)keyreversez[SEARCH] Highest score found: z.3fz[SEARCH] Match z: ':N2Nz' score=z#[SEARCH] No match above 0.25 for: ''z?[CASE 4] TRIGGERED: No DB match + No conversation history for 'a#I don't have reliable information about this specific topic in the available content. Could you please provide more details or rephrase your question? Alternatively, you might want to ask about related topics like general health principles, wellness practices, or preventive care approaches.rOrPrqg333333?c3,<"TF qS9xK R#5i)Nr])rwordquestion_lowers& r0rrs,nPm^-CPms:N NFz4[CASE 3] TRIGGERED: No DB match + Continuing topic (.2fz similarity) for 'z Matching against: Question='z ...' Answer='z...':NiNz Q: z A:c3<"Rp\pV!S4F=pVP4'gKW, pVx\P!R4K? RP VP 44p\ SPSPV4\P!4S, p\RVR R24R# \dpR\T4 2xRp?R#Rp?ii;i5i)rr{Gz?rqz[TIMING] CASE 3 LLM: rsrN) rr{rTsleepjoinsplitrVrDrOrr&rF) full_answer llm_functionr clean_answerllm_timer/ llm_prompt llm_startrs r0stream_case3_response#chat..stream_case3_responses"$K7'@ %1*%=E${{}} + 4 &+ $ 4 0 &>(+xx 0A0A0C'D 3A4E4EqzzS_`#'99;#: 5hs^1EF$7 -c!fX6667s4C-$CBCC- C*C%C-%C**C-z text/plain) media_typez3[CASE 4] TRIGGERED: No DB match + Different topic (z Context: Question='c3<"SP4p\V4F>wrY!\V4^, 8dRMR,x\P!R4K@ R#5i)rrqrrrN)r enumerater(rTr)wordsrrgeneral_answers r0stream_general_response%chat..stream_general_responsesK"((*E$U+Ua%7cR@@ 4 ,sA A#rz7SELECT content FROM dr_young_all_articles WHERE id = %sr z [CONTEXT] z articles matched:z [z ] Title: z URL: z Score: z u' [CONTEXT → MODEL] Total Characters: z | Sent to LLM: :NipNz [QUESTION] z[FULL CONTEXT FROM DB]:z [END CONTEXT]a`<|system|> You are a strict Q&A assistant for Dr. Robert O. Young's alkaline lifestyle research. RULES: 1. ONLY use information from the CONTEXT below. Do NOT add outside knowledge. 2. If context lacks the answer, say: "I don't have enough information based on the available articles." 3. Do NOT invent, guess, or fabricate any facts not in the context. 4. Start your answer directly. No preambles like "Sure!", "Based on the context..." etc. 5. Use professional scientific tone. Be concise and accurate. 6. Use numbered lists (1.) or bullet points (-). Each item on its own line. 7. Always spell "Zeolite" (no accents) and "pH" (lowercase p, uppercase H). 8. CRITICAL: When the context labels something as "FALSE" or a "myth", it means that claim is WRONG. Do NOT present false/myth claims as facts. Instead explain why they are wrong according to the context. 9. Dr. Young PROMOTES the alkaline lifestyle. All answers should reflect his pro-alkaline position as stated in the context. 10. Do NOT recommend fruits, dairy, sugar, or acidic foods as healthy unless the context explicitly says so. <|user|> CONTEXT: z QUESTION: z <|assistant|>c 3< "Rp\pV!S4FOpVP4'gKVPRR4pW, pVx\P!R4KQ .p\ 4pSFRpVR,P4P 4pWd9gK0VPV4VPV4KT V'dXRxVF#pVR,'gKRVR, R2xK% R x\V^4FwruV R VR, R2xK R PVP44p\SPSPV4\P!4S, p \P!4S, p \R 4\R SR RSR RSR R24\RS R RV R RV R R24SP%4S P%4R# \ dp R\#T 4 2xRp ? LCRp ? ii;i R#;i SP%4S P%4i i;i;i5i)rrrtrurr z See here for more info: rz- rz References: z. rqz [TIMING]:z Embedding: rzs | DB: z s | Search: rz Context: z s | LLM: z s | Total: rN)rr{rzrTrsetlowerr%addrrrrVrDrOrr&rFr )rrrunique_references seen_titlesref title_cleanrrr total_timer/r) context_timer*db_time embed_timerr8r references search_time start_times r0stream_responsechat..stream_responsehs / -L%f-;;==!MM*i@E(KKJJt$ .!# %K!!'l00288: 1%,,S1OOK0 " !55,C5zz "3u:,b11-(''(91=FACr#g,r22>88K$5$5$78L '(9(91::| Tyy{Y.Hz1J +  K 3/x}LQ\]`Paabc d Il3/y#kR\]`Qaabc d    +!#a&* * +       s}I$HA1H&-HH.C0H H)?I H& H!H1!H&&H1)H.+I1I3 IIIIIz<============================================================u────────────────────────────────────────────────────────────)rTrMrDrr( embed_modelencoderOrrr'rlrrrsumminr%sortedrr{rr intersectionunionanyrrfetchoner|r)5rrU embed_start query_embdb_startscored search_start highest_scoreartscore skip_wordsrmeaningful_words title_matchescontent_matchesr unique_scoredr title_keyrlast_interaction last_question last_answercombined_contextconversation_keywordscurrent_keywordsoverlap total_uniquesimilarity_ratiorr context_start context_parts_ content_rowcleanedcontextr r)rrr*rrrrrr8rrr r rs5f @@@@@@@@@@@@@@@r0chatr4iss J'q'8'89G $Q%6%6$7 89 s7|nM :;))+K"" #I {*Jyy{H  D +++ &CiikH$GF99;LM}yk"23LfLeLULDL%LLdLFLDLRWLY_LaeLgkLmqLsxLz~LAFLHNLPVLX\L^aLcgLimLouLw@LBKLMTLV\L^dLfnLpvLx~L@GLIPLRXLZ`LbiLkrLtzL|CLEKL '(zz'7'7'9'?'?'Ah'A!jEXA]`ab]cfg]gAA'Ah g,,,.K WWY/552u=CCEM P+;PPM!!T-=!TTO S-t4 4E S4/6 6E  !M 4< MM5, '7<F =F%KM3L&&(..0 K ' OOI &  !S * }  "  F))+ ,K *=*= >? (3OA| OA3c#g,s*;)(>(@$A !">#7#7#9: /<<=MNOG4::;KLML9E9Iw5q  #%,nP`PfPfPhilPm,n,nP`PfPfPhilPm,n)n)nLM]^aLbbtuvuuuAABCD7 c8J7K=YdehYiXjjnop#.d"3!45JJ< !IIK 7,))>)@\ZZKL\]`Kaastut~t~s@ABC.}S/A.B-P[\_P`OaaefgI !!!8!:|TTIIKMMJ3 S\ 3u:  MPSTXPY|\lln ;#K $:;G   ) Bvh- Js=)**< =>$VQ/rbsysosrTrvr#r; subprocessnumpyr! collectionsrasynciodotenvrfastapirfastapi.responsesrpydanticrr=rrrpathdirnameabspath__file__ current_dir project_rootinsert database.dbrapptorchwarningsloggingenvironfilterwarnings FutureWarning getLoggersetLevelERRORrrrLr'r1 threadingThreadstartrArMrVrXrlr|rrr>r4r]rIr0rTs  /5./ggoobggooh78 ww{+ < & 0T  '. #$), %&=9)*33GMMB =>!  &''( 4< *48>>@T4 ,T:@@B0 4 %) % B:S