Equilibrando precisão e desempenho: como os novos parâmetros do Zilliz Cloud ajudam você a otimizar a busca vetorial

Introdução

Nos últimos anos, grandes modelos de linguagem (LLMs) e bancos de dados vetoriais tornaram-se a espinha dorsal de inúmeras aplicações impulsionadas por IA — de recomendações de e-commerce a reconhecimento facial e sistemas RAG. No entanto, esses diversos casos de uso apresentam um desafio comum: como equilibrar a precisão da busca (taxa de recall) com o desempenho (latência e throughput)?

É exatamente por isso que estamos animados em apresentar dois novos recursos poderosos na versão mais recente do Zilliz Cloud, nosso banco de dados vetorial totalmente gerenciado, construído sobre o Milvus:

• O level parâmetro - Um ajuste simples, mas poderoso, para refinar a precisão da busca

• O enable_recall_calculation parâmetro - Uma ferramenta integrada para estimar e validar taxas de recall

Essas adições capacitam os desenvolvedores a encontrar o equilíbrio perfeito para seus casos de uso específicos — seja você precise de recomendações extremamente rápidas ou de aplicações de segurança altamente precisas. Neste blog, mostraremos exatamente como aproveitar esses recursos para otimizar suas implementações de busca vetorial.

Diferentes casos de uso, diferentes requisitos

À medida que a busca vetorial se prolifera entre os setores, observamos que os requisitos de recall, latência e consultas por segundo (QPS) variam drasticamente entre as aplicações. Vamos dar uma olhada em dois exemplos contrastantes:

Sistemas de recomendação: velocidade acima da precisão perfeita

Sistemas de recomendação filtram vastas bibliotecas de conteúdo para sugerir itens relevantes com base nas preferências do usuário. Nesses sistemas, a taxa de recall não é a principal prioridade. Embora as recomendações devam ser relevantes, introduzir alguma variedade frequentemente melhora a descoberta e o engajamento do usuário.

Em vez disso, esses sistemas devem lidar com milhares de solicitações simultâneas em tempo real, exigindo:

• Alto QPS para atender muitos usuários simultaneamente

• Latência muito baixa para experiências de usuário responsivas

• Recall moderado (85-95%) com tolerância para algumas correspondências imperfeitas

O impacto comercial de recomendações lentas geralmente supera sugestões ocasionalmente imperfeitas, tornando a otimização de desempenho crítica.

Reconhecimento facial: a precisão é inegociável

Sistemas de reconhecimento facial, especialmente em contextos de segurança, têm requisitos totalmente diferentes. Eles devem identificar com precisão usuários autorizados para evitar tanto falsos positivos (violações de segurança) quanto falsos negativos (usuários legítimos sendo negados).

Esses sistemas precisam de:

• Recall muito alto (99%+) para identificação precisa

• Tolerância para latência moderada (usuários aceitam um breve atraso de verificação)

• Demandas de QPS mais baixas (a verificação é uma tarefa relativamente infrequente)

As consequências de uma identificação incorreta são significativas, tornando a precisão a prioridade inegociável, mesmo ao custo de algum desempenho.

O fio condutor: encontrar seu equilíbrio

Esses exemplos contrastantes destacam por que uma abordagem única para configuração de busca vetorial fica aquém. Cada aplicação se situa em algum ponto desse espectro, exigindo otimização cuidadosa de recall, latência e QPS com base nas necessidades do negócio.

É exatamente por isso que introduzimos os parâmetros level e enable_recall_calculation — para dar aos desenvolvedores as ferramentas para encontrar seu equilíbrio ideal.

Apresentando os recursos de ajuste de precisão do Zilliz Cloud

O parâmetro level: refinando a precisão da busca

O parâmetro level oferece controle simples, mas poderoso, sobre a precisão da busca, com valores variando de 1 a 10:

Valor do Nível	Ideal Para	Recall Típico	Impacto no Desempenho
Mais baixo (1-3)	Aplicações focadas em desempenho	90-97%	Impacto mínimo na latência, maior QPS
Médio (4-7)	Aplicações equilibradas	97-99.5%	Impacto moderado na latência, bom QPS
Mais alto (8-10)	Aplicações críticas em precisão	99.5%+	Maior latência, QPS reduzido

Estendemos o limite superior de 5 para 10 com base no feedback dos usuários, permitindo uma precisão ainda maior para segurança, controle de riscos e muitos outros cenários que exigem precisão.

Observe que uma taxa de recall mais alta nem sempre é boa para seus casos de uso. Por exemplo, se level=3 ou 5 satisfaz as necessidades, aumentar ainda mais o nível levaria apenas a uso desnecessário de recursos e maior latência.

O parâmetro enable_recall_calculation: Validando sua precisão

Embora o parâmetro level ajuste a precisão da pesquisa, como você verifica se está atingindo sua taxa de recall alvo? É aí que entra enable_recall_calculation.

Quando ativado durante uma operação de pesquisa, este parâmetro:

• Estima a taxa de recall real da sua configuração atual

• Retorna esse valor junto com seus resultados de pesquisa

• Permite decisões orientadas por dados sobre mudanças de configuração

Este cálculo único ajuda você a validar se suas configurações atuais atendem aos seus requisitos de precisão sem exigir ferramentas externas de benchmark.

Implementação prática: um guia passo a passo

Vamos percorrer um exemplo real de otimização de uma implementação de pesquisa vetorial com esses novos recursos.

Pré-requisitos

• Uma instância do Zilliz Cloud (inscreva-se aqui se ainda não o fez)

• PyMilvus 2.5.4 ou posterior

• Seu conjunto de dados de teste (usaremos o conjunto de dados cohere-1M neste exemplo)

Etapa 1: Defina seus requisitos

Antes de mergulhar no ajuste de parâmetros, defina claramente seus requisitos:

• Taxa de recall alvo: 99.5%

• Latência máxima aceitável: 5ms

• Outras considerações: QPS, utilização de recursos

• Padrão de consulta: pesquisa vetorial pura

• TopK: 16,000

Etapa 2: Importe dados de teste

Para testes precisos, você deve usar dados que se assemelhem bastante ao seu ambiente de produção. Neste exemplo, usaremos o conjunto de dados cohere-1M e o importaremos para o Zilliz Cloud via VectorDBBench.

Etapa 3: Estimando a taxa de recall de linha de base

Vamos primeiro verificar como as configurações padrão (level=1) se comportam:

search_params = {
    "params": {
        "level": 1,
        "enable_recall_calculation": True
    }
}

res = client.search(
    collection_name = "ZillizCloudVectorDBBench", 
    # test data 
    data = [
        [0.22252834,0.26758388,0.3414864,0.31775144,0.25819996,-0.06176423,0.60313016,-0.31930527,-0.05070293,0.80085576,-0.7066278,-0.14704825,0.07324219,-0.051405758,0.24823247,0.20365287,-0.005265507,0.24754052,0.06302843,-0.24397966,-0.2805658,0.543768,0.018544307,0.14154078,0.03093845,-0.25058296,-0.61569184,0.08389459,-0.27519965,0.2121497,0.26527727,-0.03291658,-0.2631627,0.026973603,-0.22165383,0.38862047,0.012616088,-0.066382475,-0.013436819,-0.59001106,-0.08682751,0.13704056,0.08583454,0.0802483,0.01096984,0.20214474,0.11156094,0.5482859,-0.0807617,-0.16539982,-0.29261217,0.08269717,0.03385099,-0.48874223,-0.013168857,0.01616468,-0.6270225,0.13169415,1.0166928,0.6573267,0.40487188,0.2235163,-0.68331105,-0.24911362,-0.1763628,0.34692895,0.077760294,0.96388775,-0.10841275,0.3977706,0.08965021,0.29019687,0.106024966,0.11854912,-0.070255764,-0.24960922,-0.5354312,-0.70186704,0.25364435,0.43369204,0.42516047,0.15078346,-0.35151976,0.4886603,0.37026608,0.39485633,-0.046821307,0.18807457,0.13850202,-0.15630293,0.1469321,0.13219471,-0.31201053,0.22278261,-0.23063508,0.42379102,0.66762435,-0.11903275,0.22101034,-0.10102379,0.21675844,-0.2571628,-0.26546052,0.6185626,0.241754,-0.15792729,-0.37045696,0.23996626,-0.27011713,-0.11512929,0.31577003,0.11118326,-0.76019096,-0.1682175,0.5329204,0.3614348,-0.2348311,0.108711846,0.068240434,-1.3124024,0.20385003,0.6049856,-0.2924265,-0.07293733,0.5181119,0.15742214,0.7537572,-0.656809,0.57201636,0.09775318,0.5414663,-0.53804034,-0.0802571,0.62367904,0.023681821,0.6950041,0.3207407,0.36089638,0.53875273,-0.7288189,-0.12956187,0.15076943,0.057313107,0.41065657,-0.00928412,0.090415776,0.18091775,-0.010793066,-0.010142676,-0.20986095,-0.3740831,0.25086942,-0.31494206,0.17761512,0.04850758,-0.06098805,0.7605751,0.3038707,0.5178377,-0.5769507,0.80365956,0.22879237,-0.34868854,-0.2688102,-0.20910782,0.3392469,0.2990533,0.5502763,0.6561665,0.04177933,-0.45408615,-0.055697974,0.05596695,-0.22720425,0.63778013,-0.1921504,-0.16227654,-0.053658817,0.04536426,-0.28570235,0.30350783,0.5217574,0.0025516534,0.10135456,0.5973671,0.09276529,0.7803261,0.45648357,-0.21722879,-0.3496141,0.18574907,0.1729008,0.6754883,-0.5101994,0.16308193,-0.32053986,-0.0013728795,0.0371755,-0.114131995,0.19870742,0.3973309,0.17016156,0.016581284,0.3074155,0.32889378,-0.6682561,0.36933577,0.45571226,-0.19315217,-0.5065343,0.15996625,0.026897952,0.046443015,0.2667398,-0.18946062,-0.4283052,-0.44281873,-0.24062063,0.41703427,-0.30064407,0.35975343,0.31060407,0.18125875,0.14912511,0.10962614,-0.06901708,-0.2846222,-0.027887726,0.037055127,0.031954445,0.56672156,-0.0863331,0.1497875,-0.1635759,-0.25121027,0.6303942,0.17385906,0.4313834,0.15800661,-0.6267578,-0.03539913,0.32520285,0.42759246,0.24401832,0.2115575,-0.8652025,0.13317755,-0.5719402,0.17294376,-0.12595764,0.34818307,0.24802469,0.05904272,0.1538172,-0.57994705,0.2582915,0.45511153,-0.44164076,-0.074042775,0.04943926,-0.1648779,0.3280813,0.5601293,-0.0018850226,-0.140464,-0.07845455,0.44026145,0.56197315,0.1462102,-0.18595229,0.014953136,0.46956787,-0.14819877,0.1859354,0.019512085,-0.01712815,0.5366789,0.7835224,-0.7423546,0.6503855,0.44647282,0.3631722,-0.66614413,0.10151727,-0.1348695,0.32992417,0.10387001,-0.26746857,-0.33413792,-0.5662058,0.36110422,0.7741211,-0.039930806,-0.15249825,0.09454683,0.4891987,0.0062028184,0.06745152,0.55465925,-0.06739082,0.5588079,-0.43696547,0.555966,0.56702,0.056295626,-0.62005293,-0.3722073,0.21030217,-0.017268468,0.95288086,0.51696795,-0.25066343,-0.3169728,0.42543235,0.31396082,0.17551036,0.3922707,0.07407632,0.91187936,0.38888615,-0.12070266,0.011815081,-0.45720986,0.04727247,0.62094855,-0.45443076,0.16062841,-0.40287957,-0.55417335,-0.3830013,0.055438586,0.1718703,-0.6422826,0.22917171,-0.5290951,0.1585279,0.07934802,0.50577295,-0.035466444,0.088082545,0.5693788,0.11773129,0.1821725,0.41347143,-0.2278498,0.50422746,0.29794943,-0.9369089,0.47065943,0.28594327,-0.6866015,-0.63375616,-0.15243755,-0.46409172,-0.4630497,-0.25025153,0.6375835,0.54886156,0.19831929,-0.03725618,0.20592122,0.36338213,0.31409082,-0.05410012,0.14887711,0.09740482,0.05067692,0.14775206,0.28025475,-0.34377113,-0.27423778,-0.354568,-0.20043556,0.3899774,-0.19812085,-0.36292082,-0.18255037,0.07038237,0.642794,-0.060884897,0.2948623,0.68766963,0.6928454,-0.3849391,0.17996079,0.12549743,0.10299729,-0.25861496,-0.09246836,-0.32353002,-0.01378604,-0.095313616,-0.04558251,0.20014873,-0.4066689,-0.08052519,-0.4618455,0.37693843,0.45283204,-0.114583425,0.050728872,0.13196129,-0.1941961,-0.11727777,3.9586966,0.05150596,0.11701303,0.5739518,0.07567582,0.48247826,0.25156844,0.38180268,0.12796494,0.009531708,-0.04081167,-0.30954623,-0.035167653,0.43064785,0.24091315,-0.11113215,0.027972942,0.3501582,0.54151994,0.14281327,-0.6420307,0.48611403,0.5221564,0.47878447,0.8510151,0.5528693,0.27463847,0.7548287,0.760392,0.4057206,0.5247366,0.6815815,0.46189928,-0.0665814,0.29575244,-0.13240056,0.44400057,-0.26975283,-0.15510945,0.15475176,-0.46221858,0.054507546,0.14640503,0.66453534,0.19300742,-0.3626597,-0.16279799,0.3795997,0.122737944,-0.20419496,0.18285695,0.027228406,-0.22584598,-0.16478994,0.28747237,0.53937024,0.44095138,0.6340223,-0.41380823,0.38367343,0.39497304,-0.043954037,0.38885015,-0.33315817,-0.4766579,0.17371525,-0.23392603,0.7948543,0.3054392,-0.72041094,0.2532946,0.415873,0.80443436,-0.34634262,-0.4886025,0.30351955,-0.049782824,-0.47253707,-0.11401102,-0.096243046,0.19083612,-0.34427363,-0.24545296,0.5773733,0.16357873,0.38620606,0.39995435,-0.65907687,0.6957725,0.24120355,0.34054404,-0.039899644,0.80393964,0.06337182,0.14144897,0.117613785,-0.019442292,-3.7490542,-0.38971332,0.14894387,-0.61240107,0.19039957,0.23817067,0.022639165,0.015894404,-0.6198486,0.14320132,0.041371442,-0.30882874,-0.30676636,1.0463533,-0.034157425,0.31748047,0.4891939,0.5333419,-0.3289819,0.14962271,0.2807266,0.35519713,0.4001028,-0.18559772,-0.7066097,0.14664957,-0.565848,0.013109448,-0.18452193,-0.07372118,0.28156808,-0.36035228,0.8867393,-0.16306667,-0.04191513,0.4594507,0.43135175,-0.091903865,-0.042651527,-0.32555583,-0.19054003,-0.06525034,0.16911364,0.04686202,-0.038171988,0.1336097,0.3761719,-0.050084345,-0.2679286,0.64759475,0.7107872,0.2074471,-0.27312976,0.40090975,0.5491712,-0.10747743,0.74496686,0.18130445,-0.09431538,0.19524746,-0.21418755,-0.12488151,0.15227054,-0.3852693,-0.7784234,-0.14571632,0.041122317,-0.16407914,0.03949264,-0.1925929,0.32901394,0.12069722,0.23391949,-0.16763128,-0.12962814,0.5088096,0.21486548,-0.20993523,0.603585,0.24633685,-0.14029086,-0.27401388,-0.49189645,-0.10249644,2.3196032,-0.12417316,2.1448603,0.058190174,-0.6551869,0.6827868,0.6356786,0.7710372,0.3722568,0.8363127,0.3799041,0.26085538,-0.20764771,0.512162,0.08349497,-0.15835808,0.5738307,-0.66654295,0.18993358,0.32188657,0.0764867,0.64592606,-0.2310478,0.18350935,-0.3915338,0.028645294,-0.101273224,0.8696747,-0.50792813,-0.39119712,-0.30162883,0.7319297,0.71813834,0.39383802,-0.012138247,0.3298783,0.23386809,4.5470805,-0.049004212,0.107414484,0.052308656,0.2678271,-0.15366946,0.5438965,-0.47809094,-0.14649442,0.022792917,0.1358324,-0.4503206,0.57014287,-0.13368002,0.23805767,-0.22125027,-0.08700341,0.045676652,0.16678812,-0.27974084,0.45245427,-0.2107062,0.6667994,0.036875203,0.54632777,0.20104687,0.5349449,0.06913179,-0.086024776,0.76876926,0.16203642,5.155099,0.2797164,0.21450946,-0.17529553,-0.038863413,0.5156995,0.08603405,-0.516439,-0.35604522,0.10131945,0.008194211,0.084706515,-0.34049395,0.21572115,-0.83385843,-0.046860088,-0.48247585,0.023293016,0.22008015,-0.5305121,0.5061096,0.0183293,0.1326365,0.22057603,-0.43027383,-0.3885953,0.1500542,-0.1449458,-0.38747045,0.2789606,0.27069542,-0.37978157,-0.58541,0.5139468,-0.60586643,-0.5236463,0.22003366,0.15764758,0.3512009,0.13694952,0.7772281,0.28431293,0.113065295,0.14233269,-0.047996823,0.0024461043,0.06218189,-0.28065726,-0.2061346,-0.36278206,0.24291486,-0.0869041,0.7448049,0.36513415,0.61559093,0.42820337,0.41123256,-0.32082868,-0.10876272,-0.028618973,0.6750199,-0.048880983,-0.12521495,0.1926665,0.6695621,0.21937566,0.46856737,0.30544627,0.2650348,-0.11578811,-0.15696093,-0.047148716,0.19283816,0.12149068,-0.03274016,0.021503512,0.008024155,0.19709297,0.15727529,0.14134975,-0.16997191,-0.063695885,-0.39591065,-0.11891319,-0.04673462,0.16978487,-0.09345571,0.11924938,0.13301763,-0.2266567,0.4164705,0.3571622,0.09038913,0.18044233,0.09119875,-0.23754075,0.45051736,0.35435763,0.20957275,0.5704436,-0.36682,0.26963162,0.15532929,-0.24306794,0.17486432,0.39116114,0.12234816,0.21448524,-0.019066956,-0.09756305,0.4465544,0.3394048,-0.7088385,-0.5032021,0.03529406]
    ], 
    limit = 16000, 
    search_params = search_params
)

print(f"recall: {res.recalls}")

A saída é:

recall: [0.9886875152587891]   

Com um recall de 98,87%, estamos abaixo da nossa meta de 99,5%. Vamos medir a latência também (opcional):

search_params = {
    "params": {
        "level": 1,
#        "enable_recall_calculation": True
    }
}

start_time = time.perf_counter()
client.search(collection_name = "ZillizCloudVectorDBBench",
    data = data,
    limit = limit,
    search_params = search_params
)
end_time = time.perf_counter()

elapsed_time = end_time - start_time
print(f"latency at level=1: {elapsed_time:.2f} seconds")

E a saída é:

latency at level=1: 0.03 seconds

Etapa 4: Encontrar o valor ideal de Level

Agora, vamos testar sistematicamente diferentes valores de level para encontrar a configuração ideal:

recall at level=1: [0.9886875152587891]
latency at level=1: 0.03 seconds
...
recall at level=6: [0.9947500228881836]
latency at level=6: 0.04 seconds
recall at level=7: [0.9961249828338623]
latency at level=7: 0.04 seconds
...
recall at level=10: [1.0]
latency at level=10: 0.06 seconds

Os experimentos mostram que level=7 atende à nossa meta de recall (99,61% > 99,5%) mantendo a latência dentro de limites aceitáveis.

Etapa 5: Monitorar o desempenho do sistema (opcional)

O Zilliz Cloud oferece um conjunto abrangente de métricas do sistema para ajudar você a monitorar o desempenho de forma eficaz. Ao ajustar o parâmetro level e aplicar novas configurações, o Zilliz Cloud fornece acompanhamento em tempo real de métricas-chave como QPS e latência. Isso permite avaliar o impacto dos ajustes de parâmetros no desempenho geral do sistema, possibilitando uma tomada de decisão mais informada.

Etapa 6: Validar com carga de trabalho de produção

Para validação em produção, recomendamos:

1. Executar testes com vetores de consulta representativos

2. Monitorar o painel do Zilliz Cloud para:

   • Latência média

   • QPS

   • Utilização de recursos

Etapa 7: Realizar benchmarking abrangente (opcional)

Para uma avaliação mais rigorosa, usamos o VectorDBBench, uma ferramenta de benchmarking de código aberto, para testar várias configurações com 1.000 operações de busca:

level	Latência média(ms)	QPS	Recall do VectorDBBench	Recall do Zilliz Cloud
1	3.1	1266	0.9519	0.953
2	3.2	1080	0.9644	0.9669
3	3.4	972	0.9728	0.9755
4	3.6	814	0.9816	0.9846
5	3.9	704	0.9871	0.99
6	4.4	549	0.9916	0.995
7	5	448	0.9936	0.9971
8	5.4	375	0.9945	0.9983
9	5.9	340	0.9952	0.9991
10	6.3	296	0.9958	0.9995

Tabela 1: Resultados dos testes com base em 1.000 operações de busca do VDBBench

Esses resultados de benchmarking confirmam que:

1. As estimativas de enable_recall_calculation correspondem de perto ao desempenho real de recall

2. Um recall mais alto vem ao custo de maior latência e QPS reduzido

3. Para nossos requisitos-alvo, level=7 oferece o equilíbrio ideal

É importante observar que melhorias no recall geralmente vêm ao custo de maior latência e QPS reduzido. Se o recall atual atende às suas necessidades de negócio, mas o QPS está abaixo do ideal, você pode considerar aumentar os recursos de CU ou adicionar réplicas para melhorar a taxa de transferência.

Etapa 8: Confirmar a configuração ideal

Com base nos resultados experimentais, a configuração ideal neste exemplo é level=7, que oferece 99,6% de recall mantendo uma latência aceitável.

Conclusão

Encontrar o equilíbrio perfeito entre precisão de busca e desempenho há muito tempo é um desafio para os usuários de bancos de dados vetoriais. Com os novos parâmetros level e enable_recall_calculation do Zilliz Cloud, os desenvolvedores agora têm ferramentas poderosas para otimizar suas implementações de busca vetorial de acordo com seus requisitos específicos.

Seja criando sistemas de recomendação que priorizam velocidade, aplicações de segurança que exigem alta precisão, ou qualquer coisa entre esses dois extremos, esses recursos permitem que você:

1. Ajuste com precisão a exatidão da busca para atender aos seus requisitos

2. Valide as taxas reais de recall com estimativa integrada

3. Tome decisões baseadas em dados sobre compensações de configuração

Estamos comprometidos em continuar aprimorando o Zilliz Cloud com recursos que tornam a busca vetorial mais poderosa, flexível e fácil de otimizar. Esses novos parâmetros são apenas o começo da nossa jornada para capacitar desenvolvedores a criar aplicações excepcionais impulsionadas por IA.

Pronto para começar a otimizar suas buscas vetoriais? Cadastre-se no Zilliz Cloud hoje mesmo ou faça login na sua conta existente para explorar esses novos recursos.