Tag Archives: ai

[javaone 2025] next level features of langchain4j

Posted on by
Reply

Speakers: Lize Raes, Mohamed Abderrahman

See the table of contents for more posts

Concepts

  • SystemMessage – instructions
  • ContentRetriever – context
  • Tools – function calling
  • UserMessage – user to LLM
  • AiMessage – LLM to user
  • ChatMemory

5 Levels towards AGI

  • Can perform work of entire orgs of people
  • Can create new innovations
  • Can take actions on users behalf
  • Can solve basic problems like a PHD with tools
  • Current AI like ChatGPT that takes with humans

Options

  • LLM manages step transitions in state machine – can jump states when unexpected requests, flexibility, but risky
  • Code manages step transitions – any complexity possible, reliable, separation of concerns, tailored model size. However, not flexible. can’t deal with unexpected scenarios and more work to write.

RAG

  • Retrieval Augmented Generation
  • Fetch info relevant to request and send information to LLM
  • Advanced RAG features
  • Retrieval Augmentor in addition to retriever
  • LLM writes query
  • Adds info/context
  • Need to measure performance of model. Compare across models
  • MCP (Model Context Protocol)

Steps in code:

  • Create document content retriever – can limit scope. Ex: scientific literature
  • Create web search content retriever
  • Create SQL database content retriever

Guardrails and Moderation

  • Guardrails add limits. Ex: list examples of queries that shouldn’t be allowed
  • Moderation – checks if message violent, etc. Can use a different model for validations
  • LLMs are more sensitive to examples than instructions

Testing approaches

  • Test human evaluation (thumbs up/down)
  • AI assisted

Websites

  • swebench.com – closes github issues
  • llm-price.com – shows prices per token and per million tokens
  • JUnit Pioneer – test retry
  • Examples from session: https://github.com/LizeRaes/ai-drug-discovery

My take

Excellent examples. The real world scenario of diseases/antigens/antibodies was good. Good concepts and great demo. Showing Prometheus/Graphana was good as well.

Free Copilot.. Which IDEs shall I use

Posted on by
Reply

Earlier this month, Microsoft announced free GitLab Copilot. You get 2000 autocompletes a month and 50 chat messages. The idea is clearly that you’ll start relying on it and then pay.

I installed and configured in both VS Code and IntelliJ. Setup was easy. I had to authorize the IDE by entering a code on github.com that the IDE gave me. It also gave me the link to enter it so pretty easy.

I also installed it on PyCharm and quickly uninstalled it. Copilot was being too helpful and I need to practice more typing the idioms as I get better at Python. (This is my fifth time learning Python. Every time I get decent at it, I don’t use it for a long time and forget. Maybe this time it’ll take!)

I do use AI periodically to ask how to improve the Python code I write (at home on personal stuff). But I can do that outside my IDE.

I like that Microsoft is doing this. They’ve had a free trial of CoPilot for a long time, but that is time limited. I like that this one is usage limited. So if you don’t use it much, you have it when you need it and still free.

[2024 dev2next] Breaking AI

Posted on by
Reply

Speaker; Micah Silverman @afitnerd)

For more see the table of contents

Notes

  • ChatGPT took 2 months to get to 100 million global monthly active users. By contrast, TikTok took 9 months, Uber 70 months, Instagram 30 months
  • Hot trend, but also people found utility in it.

App Security

  • Getting hard.
  • Code growing faster and apps getting more complex

Common uses in dev

  • Adding comments
  • Summarizing Code
  • Writing “readme”
  • Refacotring code
  • Proividing templates
  • Pair programming
  • Generating code – the new stack overflow

Stats and studies

  • 92% using AI coding use
  • 57% completed tasks faster (not necessarily better)
  • 27% more likely to finish task
  • 40% co-pilot code contained vulnerabilities
  • More likely to believe wrote more secure code, but wrote less secure code. Because believed was more secure, didn’t look hard.

AI code

  • Like junior dev just out of bootcamp. Need to checked works and secure
  • Example hallucinations. Change over time. Over a few months, went from 98% on math to 2% on math. Open AI fixed basic math. Designed to be good prediction engines, not math
  • “chatGPT is confidently wrong” – Eelko de Vos

AI Coding

  • Asked for an Express app to take name in request param and returns a website showing name
  • All LLMs tried had XSS/injection
  • If questioned or asked to create a secure express app, would get sanitized one. Let of sanitization varies.
  • Showed Synyk advisor – gives health score on libraries – ex: sanitizer. Need to check recommended libraries

Co-pilot

  • 40% code trained on is insecure
  • Used approach where prompt through comments (vs chat feature)
  • Used live templates to autocomplete prompt comments to save time for demo
  • Example with Spring boot and Thymeleaf. Copilot got that from context of project
  • Not quite right but made minor changes vs starting from scratch
  • Copyright was 2017; noted hadn’t done that before
  • Copilot tried to provide the next comment/prompt. Not what wanted, but reasonable
  • Synk IDE extension – Detected SQL injection in view that looks like problems view

Chat GPT

  • Had do a security code review
  • Added HtmlUtils.htmlEscape(username) – context aware; knew using Spring Boog

My take

Micah said up front that he has no connection to Microsoft or IntelliJ and is just using their products. I never thought to give that disclaimer when I use tools. I’ll think about whether I want to when it isn’t almost 9pm. I am very much a morning person. In fact, that’s why I chose this talk. I thought it would require loading the least info into my mind to understand at this hour while still learning. The demo of copilot for building an app was fun with a good emphasis on security.