CTIBench: Evaluating LLMs in Cyber Threat Intelligence with Nidhi Rastogi - #729

CTIBench: Evaluating LLMs in Cyber Threat Intelligence with Nidhi Rastogi - #729

Today, we're joined by Nidhi Rastogi, assistant professor at Rochester Institute of Technology to discuss Cyber Threat Intelligence (CTI), focusing on her recent project CTIBench—a benchmark for evaluating LLMs on real-world CTI tasks. Nidhi explains the evolution of AI in cybersecurity, from rule-based systems to LLMs that accelerate analysis by providing critical context for threat detection and defense. We dig into the advantages and challenges of using LLMs in CTI, how techniques like Retrieval-Augmented Generation (RAG) are essential for keeping LLMs up-to-date with emerging threats, and how CTIBench measures LLMs’ ability to perform a set of real-world tasks of the cybersecurity analyst. We unpack the process of building the benchmark, the tasks it covers, and key findings from benchmarking various LLMs. Finally, Nidhi shares the importance of benchmarks in exposing model limitations and blind spots, the challenges of large-scale benchmarking, and the future directions of her AI4Sec Research Lab, including developing reliable mitigation techniques, monitoring "concept drift" in threat detection models, improving explainability in cybersecurity, and more. The complete show notes for this episode can be found at https://twimlai.com/go/729.

Jaksot(781)

Engineering the Future of AI with Ruchir Puri - TWiML Talk #21

Engineering the Future of AI with Ruchir Puri - TWiML Talk #21

Today we bring you the second of three interviews we did backstage from the NYU FutureLabs AI Summit, this time with Ruchir Puri. Ruchir is the Chief Architect at IBM Watson as well as an IBM Fellow. ...

28 Huhti 201720min

Selling AI to the Enterprise with Kathryn Hume - TWiML Talk #20

Selling AI to the Enterprise with Kathryn Hume - TWiML Talk #20

This week's guest is Kathryn Hume. Kathryn is the President of Fast Forward Labs, which is an independent machine intelligence research company that helps organizations accelerate their data science a...

21 Huhti 201723min

From Particle Physics to Audio AI with Scott Stephenson - TWiML Talk #19

From Particle Physics to Audio AI with Scott Stephenson - TWiML Talk #19

This week my guest is Scott Stephenson. Scott is co-Founder & CEO of Deepgram, which has developed an AI-based platform for indexing and searching audio and video. Scott and I cover a ton of interesti...

14 Huhti 201756min

(5/5) AlphaVertex - Creating a Worldwide Financial Knowledge Graph - TWiML Talk #18

(5/5) AlphaVertex - Creating a Worldwide Financial Knowledge Graph - TWiML Talk #18

This week I'm on location at NYU/ffVC AI NexusLab startup accelerator, speaking with founders from the 5 companies in the program's inaugural batch. This interview is with AlphaVertex, a FinTech start...

7 Huhti 201726min

(4/5) Behold.ai - Increasing Efficiency of Healthcare Insurance Billing with NLP - TWiML Talk #18

(4/5) Behold.ai - Increasing Efficiency of Healthcare Insurance Billing with NLP - TWiML Talk #18

This week I'm on location at NYU/ffVC AI NexusLab startup accelerator, speaking with founders from the 5 companies in the program's inaugural batch. This interview is with Behold.ai, which uses comput...

7 Huhti 201716min

(3/5) Cambrian Intelligence - Using AI to Simplify the Programming of Robots - TWiML Talk #18

(3/5) Cambrian Intelligence - Using AI to Simplify the Programming of Robots - TWiML Talk #18

This week I'm on location at NYU/ffVC AI NexusLab startup accelerator, speaking with founders from the 5 companies in the program's inaugural batch. This interview is with Cambrian Intelligence, a com...

7 Huhti 201723min

(2/5) Klustera - Location-Based Intelligence for Smarter Marketing - TWiML Talk #18

(2/5) Klustera - Location-Based Intelligence for Smarter Marketing - TWiML Talk #18

This week I'm on location at NYU/ffVC AI NexusLab startup accelerator, speaking with founders from the 5 companies in the program's inaugural batch. This interview is with Klustera, a company applying...

7 Huhti 201722min

(1/5) HelloVera - AI-Powered Customer Support  - TWiML Talk #18

(1/5) HelloVera - AI-Powered Customer Support - TWiML Talk #18

This week I'm on location at NYU/ffVC AI NexusLab startup accelerator, speaking with founders from the 5 companies in the program's inaugural batch. This interview is with HelloVera, a company applyin...

7 Huhti 201725min

Suosittua kategoriassa Politiikka ja uutiset

uutiscast
aikalisa
ootsa-kuullut-tasta-2
politiikan-puskaradio
rss-ootsa-kuullut-tasta
rss-vaalirankkurit-podcast
tervo-halme
viisupodi
rss-podme-livebox
et-sa-noin-voi-sanoo-esittaa
rss-asiastudio
otetaan-yhdet
rikosmyytit
the-ulkopolitist
rss-girls-finish-f1rst
rss-raha-talous-ja-politiikka
radio-antro
rss-kaikki-uusiksi
linda-maria
rss-sinivalkoinen-islam