Skip to main content

Trowser - Rikard Edgren | Show 032

Rikard Edgren and his new Free AI enabled Web Browser designed for software testers called Trowser

Podcast Trowser - Rikard Edgren | Show 032 Load podcast audio player
Subscribe to the show:

Watch on YouTube

Show Notes

Alan Richardson speaks to Rikard Edgren about Software Testing with AI and his new Free AI enabled Web Browser designed for software testers called Trowser.

Join Alan Richardson and guest Rikard Edgren, author of “The Little Black Book of Test Design”, as they discuss testing tools, AI in software testing, and Rikard Edgren’s innovative browser tool, Trowser. Discover insights on testing approaches, AI’s role, security, and tips to improve your own testing skills. Perfect for testers, developers, and anyone interested in the future of software quality!

Trowser - Rikard Edgren

Summary

Notable Quotes from Rikard

Transcript

Alan Richardson [00:00:00]:

All right, and I think we’re on. So another episode of the Evil Tester Show, and this time we have a guest. I like guest episodes because I have to do less work, hopefully. So we’ve got Rikard Edgren, and you’re at thetesteye.com, and you’re the author of The Little Black Book of Test Design, which I do have behind me somewhere, but you’ve got it free on your website as a PDF, so people can go off to the publication section and download that, which is a very dense, condensed book on testing techniques, thoughts, heuristics, all that kind of stuff. And you’ve also been working on a tool called Trowser, which is an AI browser tool, which we’ll discuss in a minute. And hello.

Rikard Edgren [00:00:51]:

Hello.

Alan Richardson [00:00:53]:

All right. So is that a good enough introduction? Oh no, I missed something. I missed out legendary status. From the last Eurostar conference. Eurostar legend.

Rikard Edgren [00:01:04]:

What could be worth mentioning as well is that I have been— I’ve been doing testing all the time, almost. I started in 1998, and I did one year as a project manager, and I didn’t like it, and then I went back to testing. But I’ve never become like a manager or stuff like that. I’ve been doing testing the whole time, and that’s what I like doing.

Alan Richardson [00:01:50]:

Okay, that’s cool. So, one of the interesting things about testing, just going back to what you said, when I started in testing, I started in testing because it was a way of using a huge range of skills. Like, I needed to be able to do development in testing because we had to write our own test tools because the test tools were too expensive. And although you say you I did project management, but I didn’t like it. You have no choice in testing but to do project management.

Rikard Edgren [00:02:23]:

Yeah, of course. Yeah, testing is broad, of course. Yes, but the center has always been the testing and finding out information about the sites or the software I’m testing.

Alan Richardson [00:02:33]:

Yeah, so you learn a huge number of skills even though your focus is just testing.

Rikard Edgren [00:02:38]:

Yes, of course.

Alan Richardson [00:02:39]:

Of course.

Rikard Edgren [00:02:39]:

Yeah, yeah. I wanted to become a developer. And then one company, you could start as a tester and then try to advance to developer, but I never could advance. And then I realized that I like the testing more.

Alan Richardson [00:02:52]:

So it’s certainly not that you never could advance because you’ve written multiple tools and you’re working on a tool now.

Rikard Edgren [00:02:59]:

Yeah.

Alan Richardson [00:02:59]:

So it’s that you chose not to advance into the programming space?

Rikard Edgren [00:03:05]:

Yes.

Alan Richardson [00:03:07]:

Because I, I still believe Um, in that concept of developer spanning all the roles. So that agile concept of developer where everyone is a developer, so you have a range of skills and you mix and match on a team to get the best out of more people. But we tend— we specialize to bring in that extreme domain knowledge and expertise.

Rikard Edgren [00:03:32]:

Yeah, and also, I’m— I know some programming, but I’m a weak developer. So, that was also a reason. Should I— why should I be a weak developer when apparently I’m pretty good at testing? So, that’s also into it. And now with the AI being so good at programming, we might end up with everyone being a tester and the AI can take part of the coding stuff. We’ll see.

Alan Richardson [00:03:55]:

So, certainly. Yeah, well, I mean, so I find when I’m using AI, I have to do a lot of the testing. I mean, it can write unit tests, and integration tests, and it can find issues as it’s doing that. But it is very weak at using the application to find issues.

Rikard Edgren [00:04:16]:

It depends on what your application is, I would say, and what you’re looking for. And in my experience, it’s a, it’s a great complement to the testing I do myself.

Alan Richardson [00:04:26]:

So that brings us on to Trowser. Yeah. So how would you describe Trowser?

Rikard Edgren [00:04:32]:

So Trowser is a browser specifically made for testing. So it is a standard WebView Chromium in the core, so it can show any web page. But instead of having favorites and a lot of other stuff that the browsers has, my tool has testing tools. So I have built in a lot of small tools that you want to use now and then. So you can, you can test more quickly as one example.

Alan Richardson [00:05:04]:

So this is designed to help you test rather than replace you as a tester? Yes.

Rikard Edgren [00:05:11]:

Yes. The design is to have a browser made for testing. So it’s easier and faster to do testing. And it also has a rich API, which means that an LLM can also use the browser to do the same testing that I can do. Which also means that we can work on the same browser and look at what we see at the same time. And it also means we could have 2 browsers where the AI runs one and I run one, and then we compare the results.

Alan Richardson [00:05:39]:

Or why limit to 2? 3 browsers. Yeah, multi-trousers are doing multi-user testing in on the same machine.

Rikard Edgren [00:05:47]:

Yes.

Alan Richardson [00:05:49]:

So when you say the REST API, does that mean It’s when you open up the browser, it’s got an open socket which is exposed as a REST API.

Rikard Edgren [00:05:59]:

It means that you can turn on an open socket at the port at your machine, at your localhost, and it also has an API key that’s unique for you. So it’s not, it’s, it’s not wide open for anything, but since it has an API that can do almost anything in the browser, of course it is very powerful and it’s also dangerous in that sense. If you would go to a very dangerous website, it could do other things on your machine as well, of course. So it’s not as safe as other browsers, but it’s also much more powerful. I mean, I can let the AI inject any script into the page and do anything and get access to local storage and everything.

Alan Richardson [00:06:41]:

So it’s sharing the same browser session as you are, and it’s A browser session that you have started and created.

Rikard Edgren [00:06:49]:

Yes.

Alan Richardson [00:06:50]:

Rather than letting something like Playwright open the browser, then you use it whilst it still has control.

Rikard Edgren [00:06:57]:

Yeah. So, you can say it’s like a browser plus Playwright, but it’s embedded in one, plus it also has testing tools. So, for instance, I have the axe-core library to check accessibility. It’s built in. And I built a lot of programmatic tests to check things on the page for a numerous amount of things.

Alan Richardson [00:07:20]:

Yeah, I think axe is quite good. I’ve been using axe because I’m working on other test tool experiments on the side, and one of those has got axe and Playwright and Lighthouse, and I’m just figuring out how to combine all those. I just redesigned my website because of all the axe stuff, so that might be why you didn’t find the video. On the podcast. Maybe I’ve been too excessively focused, but I remember you released a video showing Trowser, but then I couldn’t find it on your channel.

Rikard Edgren [00:07:51]:

Oh yeah, it’s— I did a, like, a demo that’s on YouTube. So if you would search for—

Alan Richardson [00:07:59]:

Have you got 2 channels?

Rikard Edgren [00:08:00]:

Uh, maybe I did a new channel called Trowser or something.

Alan Richardson [00:08:05]:

All right, so you need to send me the link to that so that I can stick it in the show notes because I had to look for it. Because I saw the Rikard Edgren channel and it had 3 videos.

Rikard Edgren [00:08:12]:

Yeah, quite old.

Alan Richardson [00:08:14]:

Right.

Rikard Edgren [00:08:15]:

Yeah.

Alan Richardson [00:08:16]:

All right, so you’re going into multiple channel mode on YouTube because you’re going to be a massive YouTube hit?

Rikard Edgren [00:08:22]:

Probably not. But anyway, I did it because to me it sounds like such a good idea, and I’m surprised we didn’t have a testing browser earlier. We have a lot of developer browsers and stuff like that. We didn’t have anything dedicated for testers. So that’s why I want to do it, and it really helps me do testing of websites faster and, and better. So the demo shows what you can do with it.

Alan Richardson [00:08:49]:

Yeah.

Rikard Edgren [00:08:49]:

One small example, just so people know how it could benefit them. In the old days, if I would try to change the request or response on the website, I would launch said attack proxy, put my browser to that proxy port, and then I would do something, make an intercept, you know, stuff like that. Now I can just open the— my network interception dialog in the browser and say, this is the regex, if this happens you should intercept. If this happens, we have this mock, I want to see how it handles the 500 error and stuff like that. So it’s, it’s, it’s nothing spectacularly new in Trowser, but it’s convenient and it combines things so you can do it faster, and then you will do it.

Alan Richardson [00:09:31]:

And you decided to wrap Chromium. Did you experiment with just creating a Chrome plugin first, or did you go straight to a browser embed?

Rikard Edgren [00:09:42]:

Uh, yeah, so the story is that someone I trusted in the autumn 2024 said that Claude Sonnet is good at programming, and I was not impressed with LLMs before that, not at all. But I trusted him, so I said, okay, let’s try it. So I started doing small test tools that helped me in my work, custom tools for the things I was testing. And it worked. It was good. It was so much fun to be able to create a tool that no one would ever create commercially, but I needed that one right now and I could create it in an hour and it helped me. So then I saw that, wow, you can do this. So I have done so many plugins.

Rikard Edgren [00:10:24]:

Maybe 50 different tools for my testing since then. And when I’d done all those, I was— in the Sonnet, you could feel that there was a limit. Maybe you experienced as well that when the projects got bigger, it had problems. Things went wrong. It couldn’t handle it with the context size and stuff like that. But then with Opus, I realized I don’t reach the ceiling. I don’t see the stop. So I was ready to try something.

Rikard Edgren [00:10:55]:

I want to do something more big than just a plugin for this. And that’s when I came up with, why not have my own browser? Wouldn’t that be cool? And then it just went on from that.

Alan Richardson [00:11:06]:

So it’s in— it’s currently Windows only. What are you writing it in? What programming language?

Rikard Edgren [00:11:11]:

It’s C#.

Alan Richardson [00:11:13]:

Okay, so in theory it could possibly run on Mac?

Rikard Edgren [00:11:17]:

Yes.

Alan Richardson [00:11:18]:

With the .NET framework?

Rikard Edgren [00:11:20]:

Um, maybe. I know it can run on Wine on Linux, so with other emulators or virtual machines it should, should be able to run on Mac as well. But I, I’ve been— you’ve been long in industry as well, so you know that if you try to do something too generic you will get into problems. So if I would have built a crossover trouser for any platform, it would have been pretty weak, would have bugs everywhere, and I wouldn’t be able to test it. So I decided from day one, this will be Windows only because then it would be something good. And also with the C# and everything, it’s fast.

Alan Richardson [00:11:59]:

And yeah, we have to make interesting architectural choices when we build this. Um, so the fact that you’re bundling it as Chrome probably gives you the ability to do things that you can’t do from an extension.

Rikard Edgren [00:12:14]:

Yes.

Alan Richardson [00:12:15]:

Um, and probably more control over the UI. Um, and it’s certainly easier to install. It’s just people get nervous installing executables where they can’t see the code, so you have to try and convince them that you’ve got a trustworthy app, but it’s an interesting experiment, but also you are using it.

Rikard Edgren [00:12:37]:

Yeah.

Alan Richardson [00:12:38]:

So it’s not just, here’s a project that I’ve created and it’s not adding any value.

Rikard Edgren [00:12:43]:

No, it’s more like I need it to do my work better. And then I’m also offer it for free to others if they want to try it. Very few others are trying it.

Alan Richardson [00:12:53]:

Extremely few. I think, well, so part of that is because Um, it’s kind of— it is hard to find. Like, you have to know the testi, um, and it’s buried as a zip on the kind of publications page at the bottom. So creating a new page might help because then the trouser browser, then you get some SEO benefits. But also that’s why we’re doing this, to try and draw some attention to it. Because I mean, I, I have something similar that’s not even close to as advanced, right? Because I wanted the ability when people are doing testing to track what you’re doing inside the browser, that concept of taking exploratory testing notes. And so I’ve got a Chrome extension which allows you to make notes in the browser, and it will do things like take screenshots, and it can do some checks and things, but it’s purely an extension, and currently it just writes out text files. And what I then do is I have the LLM point at the folder and then ask it questions about what I’ve done, what my coverage was, what ideas it’s got for something else.

Alan Richardson [00:14:03]:

You’re getting that inside the browser as you test and feedback, because I assume you can ask, um, Trowser, what else can I do? Give me some ideas about what to test.

Rikard Edgren [00:14:15]:

Yeah, I have the logging as well in Trowser, and, and Trowser itself is pure software. There is no AI at all in Trowser, but you can connect to an AI. So I have like a chat window, so if I am somewhere, I could ask the AI, what should I test here? Or I could ask it, here’s an input field, throw in all the dangerous payloads you know about to see what happens. So yeah, you can use it integrated Uh, as you want.

Alan Richardson [00:14:46]:

I mean, what I’ve also been experimenting with is the Chrome AI APIs, because inside Chrome itself there’s some AI APIs for the prompt API. So you can actually use— so I created a LinkedIn Chrome extension which, as I’m going through LinkedIn, looks at all the posts, pulls out all the ones that have got links in it, and shows me them in an extension on the side. So, because I miss too much stuff on LinkedIn, but then I use the Chrome prompt AI that’s built in to summarize those posts for me. So I’ve got summaries of all the posts that might be of interest down the side. And there’s so much you can do in the browsers.

Rikard Edgren [00:15:29]:

Yes.

Alan Richardson [00:15:30]:

And one thing I noticed with Chromium is it gives out slightly different browser headers to applications. So do you change the browser headers?

Rikard Edgren [00:15:40]:

Yeah, the— you mean the request headers?

Alan Richardson [00:15:43]:

Yeah.

Rikard Edgren [00:15:43]:

They are super important in testing, so you can change them to whatever you want. Since I own the browser, I can change anything, and the AI can change anything. And I mean, fiddling with, with headers is like the easiest way to provoke 500 errors if that’s something you, you’re interested in. And yeah, you want to change the POST body and stuff like that. And I also changed the user agent. So default it says browser.

Alan Richardson [00:16:14]:

Because I’ve noticed that some websites when I’m automating them really don’t like to be automated. So I have to change the headers in the automation in order to do the things, particularly user agents.

Rikard Edgren [00:16:27]:

Yep.

Alan Richardson [00:16:28]:

And one of the— so is this— when you’ve been building this Has this taught you more about how the web works and HTTP works?

Rikard Edgren [00:16:37]:

Oh yeah, oh yeah.

Alan Richardson [00:16:40]:

And has that then changed how you test websites?

Rikard Edgren [00:16:44]:

Yes, I would say that. Uh, I, I built it like a general testing tool for any tester who tests web services, and I’m like a broad exploratory tester. I have a look at usability and a little bit of security, not an expert, and a bit of everything, stuff like that. And then I let the AI run the API and I was astonished. Wow, it can do exploratory testing. It can test something based on the result, it does something else, it changes its mind, etc. It was fascinating. And then I realized that it was at its strongest for security testing, which it wasn’t my intention.

Rikard Edgren [00:17:21]:

It was a general tool, but I found that, wow, it can do Quite deep security, better than me. And then I needed to add some security features in Trowser because you don’t want the AI to do everything by hand because it would be too expensive. So after like a month with general Trowser, I had several months with only security, and I learned so much about security and the web and the HTTP and stuff like that. So these are like a new, new area where that I really like.

Alan Richardson [00:17:51]:

Yeah.

Rikard Edgren [00:17:52]:

to start reading on request smuggling. I mean, if you haven’t done that, wow, that’s really fascinating.

Alan Richardson [00:17:59]:

So, are you now trying to use this in bug bounties?

Rikard Edgren [00:18:02]:

I have done that, yes, but they are so slow with, uh, with the responses because they get so many things from AI. So, I’ve done it and I got a few responses that were like, yeah, thank you very much, this is really interesting, but it was not exactly what we’re looking for, and stuff like that. And I also did it on the, the VDPs Not for the money, but it can find stuff. Yes.

Alan Richardson [00:18:25]:

Right. That’s cool. When I started looking at AI, I thought, okay, this is going to impact all career choices. So maybe I should go more into security testing because I thought security testing is never going to get AI impact. And then one of the first big case studies that you see of success is security testing, where on HackerOne for several months, the top hacker was an AI bot. And it’s like, okay, we’re going to have to figure out how to differentiate ourselves as humans and add value into this process.

Rikard Edgren [00:19:03]:

Yeah, but I think your gut feeling that security testing is a good way to go, I think it’s still valid. Because you need someone to evaluate the results from the AI, and you need someone to guide them and steer them and to see where they want to look deeper. And they are really good at some things, but they’re not good at everything. But like a mediocre security tester, they need to find a different job.

Alan Richardson [00:19:32]:

Mediocre, mediocre everything.

Rikard Edgren [00:19:35]:

Yeah, a mediocre everything. need to find a new thing, something they love more, that they can learn more. So I definitely think, because the security will be more and more of a problem when it’s so easy to find the vulnerabilities, people will do it and they will do stuff. So the defensive work needs to accelerate massively for any place where your data is important.

Alan Richardson [00:19:56]:

Did you see last week, I think it was early this week or last week, Hugging Face got hacked? And then I think it was yesterday, it was said that the new OpenAI model had been the one that did the hack. You’ve got to be really careful what you prompt it with. So when you’re testing websites and you’ve got your Trowser, you’re connected to the website. How— you’re connected to the web. How are you trying to sandbox or control Trowser?

Rikard Edgren [00:20:28]:

Yeah, so The first version of Trowser, this was in February this year, it was a pure browser with nothing else, just a browser. And the very first feature I built was a scope mechanism so you could enter the domain where you want to be. So if I said I want to be on eviltester.com, it could not go outside that, it would be blocked. So that was the first feature I built because I felt This, that’s a must. If you let AI do it, it’s a must. You need me. Anything can happen. Uh, so I used it for a day or two and I never used it since then because the AI actually stays on target.

Alan Richardson [00:21:07]:

And I mean, you’re not really using, you’re not really using it as an agent.

Rikard Edgren [00:21:13]:

I have an agent working, uh, in Trowser.

Alan Richardson [00:21:17]:

So, right.

Rikard Edgren [00:21:17]:

So I haven’t, uh, I usually use Cursor. The cursor is the agent and it connects to the API and it could do anything, but it does what I tell it to do. And I’m, uh, they have probably put a lot of effort of do just what I say, don’t go away, because that’s where you get the horror stories and the bad reputation. So I have that feature to scope it, but I never use it.

Alan Richardson [00:21:42]:

And have you been experimenting with other, um, LLMs and other agents?

Rikard Edgren [00:21:49]:

Yes, I have.

Alan Richardson [00:21:51]:

And which do you— have you tried running it on local agents like Ollama?

Rikard Edgren [00:21:57]:

Okay.

Alan Richardson [00:21:57]:

And do you get similar results?

Rikard Edgren [00:21:58]:

No, my computer is not powerful enough to be able to have the model that is needed in order to do testing. I think you can do programming, which is much easier than testing, you know, but the testing is so, so difficult to do that you need, you need the Claude Opus or better. I think.

Alan Richardson [00:22:16]:

Okay, so how do you see Trowser expanding going forward?

Rikard Edgren [00:22:25]:

When I, when I built it, I didn’t know of any other way that you could go to websites, log in somewhere, and let an AI do the testing. So when I realized what I had built and what you could do, it felt, wow, this is so— wow, I’m gonna be a millionaire, was the thought I had for an hour. And then I realized that, yeah, but that’s because you like to experiment and see new things and find new problems. But most testers want to do the next Jira ticket and get the automation done. So, they won’t have use for it. I thought this will be huge, but it is not, and it will not be. It will be like a power user tool for exploratory testers who want to expand and find new things and find new ways and work a bit faster and stuff like that. I don’t think it will be a blockbuster.

Rikard Edgren [00:23:26]:

But I’m hoping that it will continue because I build new things all the time. So fast to do. So it will be new, new things, and maybe it will be a hit somewhere for someone. We’ll see.

Alan Richardson [00:23:40]:

So are you ever going to open source it?

Rikard Edgren [00:23:42]:

Well, it’s freeware now, so it’s, it’s for free. But I am right now, I’m reluctant to open source all the code because I don’t know how that would be used, and someone else might take it and sell it or make a new version that’s more of a hacker and stuff like that. I don’t know. And also, I’m not sure that I want other people to have opinions on the, the codes and stuff like that, because you know what’s, what’s taking the most time in software development?

Alan Richardson [00:24:18]:

Tell me.

Rikard Edgren [00:24:18]:

It’s, uh, it’s to agree on what to do. for people to agree what to do. That’s what takes the most time. And with Trowser, and I’m the only one, I’m the only one who have the code base, I’m the product owner, I’m the, the developer, I’m the tester, I’m everything, to agree is very fast. So I have removed that bottleneck.

Alan Richardson [00:24:42]:

So I can understand that. Um, I mean, I tend to open source a lot of my stuff But I find that no one really contributes or tries to change it anyway. And a few people do add bug fixes, which is always useful. So I guess if it becomes popular, then people might try and steer it. And, but interestingly, that might also be useful in terms of getting new features in there.

Rikard Edgren [00:25:12]:

Yeah, we, we might end up there. I don’t know. But I also have a small, like, security concern where open source is good for security if you have an active and big community, because then people will look for it and fix it. But if I open source it and no one looks at it except someone who wants to find the security flaw and exploit it and stuff like that, because since it’s a browser, it’s very powerful. So yeah, that’s also in the bag there.

Alan Richardson [00:25:42]:

Okay, um, because it’d be interesting to look at the source and compare notes with other things, um, because I mean, I’m working on so many different projects, I don’t spend a lot of dedicated time on any one. But the Chrome extension I’ve got for exploratory testing, the Observatron, I might well extend that at point, and it’d be interesting to compare notes with other tools, but that’s okay.

Rikard Edgren [00:26:14]:

I mean, I could give the source code to you, Alan, of course. That would be fine.

Alan Richardson [00:26:19]:

Well, it’s not like your Ruby one where you could actually get the source code from the executable.

Rikard Edgren [00:26:26]:

I think the new AI are really good at reversing stuff, so they probably will be able to find the stuff.

Alan Richardson [00:26:33]:

Okay, well, we’ll see how it goes on. And if you ever do want to open source it, give me a shout. But also, in terms of code quality and things, the LLM is really, really good at refactoring code once it’s given guidance on the architecture. And anytime I’m doing the AI development, I just load up the project with as many static analysis tools as I can associated with that language.

Rikard Edgren [00:26:57]:

Yeah.

Alan Richardson [00:26:58]:

get the Prettier in there for the code formatting. So I don’t do any of that stuff. And then it goes into the, the build process in Git. So it’s going through the CI process where all the unit tests are running. And the great thing is the AI will add all that for you. So have you got a lot of unit tests in your code?

Rikard Edgren [00:27:16]:

Really good question. The amount of unit tests is zero.

Alan Richardson [00:27:23]:

Is that by choice?

Rikard Edgren [00:27:24]:

Yes.

Alan Richardson [00:27:26]:

All right, why is that?

Rikard Edgren [00:27:27]:

It is because when I built my, my test tools, like, a year and a half ago, I spent a lot of time doing unit tests and integration tests, and I learned new frameworks for JavaScript and stuff like that, and it was really fun. But they never found any bugs. The bugs I found was when I used the tool for real and saw that We need to change that. So I did that. Since I’m a tester, we need to have the unit tests. Of course we should have that. And I did that. And I realized I spent more time building the tests and maintaining them than building the tool.

Rikard Edgren [00:28:04]:

And now when the AI is becoming more costly, that would mean that I would like half, or maybe I only can do one-third of my trouser feature enablements without knowing if there is value to it.

Alan Richardson [00:28:25]:

Okay. I mean, I guess since you’re still experimenting, unit tests can potentially delay it and it can slow down the development work. I do find LLMs are getting much, much better at coding. So I often use the unit tests in the codebase not so much to find bugs, but to protect it from adding new bugs when it’s working on a new area. And I find the unit tests very good at that. And also the unit tests, when I’m reviewing them, expose to me the bad architecture decisions that the LLM has put in, and then I get it to refactor as a result of that.

Rikard Edgren [00:29:05]:

So, uh, So, what I have for Trowser is I have a smoke pass. That’s actually also inside Trowser as a quick test. Trowser has like a script console and you have quick tests with code. So, it’s like it runs a smoke pass in 35 seconds or something, executing a lot of the internal mechanisms. That’s all the automated tests I have right now. I will build when I feel like, okay, now I have something. Here we are. Okay.

Rikard Edgren [00:29:33]:

This is where we— I’m not going to change a lot in this. Then I will add tests, and I right now I’m thinking about doing something that actually executes Trowser. Maybe it will be— maybe I will have a Docker instance of a web service that we will test programmatically in many ways, and we have the expected results, or an advanced internal mocking. So, so I mean, I, I’m using Trowser to execute the features to see that all of them work, something like that. But that’s like a huge effort, so that would be like the main thing to do. And right now I’m not finished with the all the nifty features of the tool itself.

Alan Richardson [00:30:16]:

I mean, that’s one of the things LLMs are really good at, experimenting and organically growing something. The danger is it organically grows a mess, so at some point You have to stop, step back, and fix the mess that has been created. Did you ever try and use OpenCode?

Rikard Edgren [00:30:32]:

Nope.

Alan Richardson [00:30:34]:

So OpenCode, if you ever get concerned about cost and tokens, is really good because OpenCode itself is a free version of like Claude or Codex. But you can sign up to OpenCode Zen, which allows you access to free frontier models. So a lot of the time when the big companies are making their models, they release them in kind of beta, which can come through open code. So I was using open code on Groq for a while. So you get access to these huge frontier models completely for free. There’s kind of limits and you’re probably helping train those models. So your code is going through to them.

Rikard Edgren [00:31:15]:

Yeah.

Alan Richardson [00:31:15]:

But if I’m working on an open source tool, I don’t really care about that. Yeah, but that’s an interesting augmentation. And I found that OpenCode is actually quite good at reviewing code. So I’ve got, um, Codex doing reviews. It’s not brilliant. I hook up to CodeRabbit. CodeRabbit is very good at doing reviews in CI, but OpenCode I found actually really good at reviewing the code base, and it’s for free.

Rikard Edgren [00:31:43]:

Okay.

Alan Richardson [00:31:44]:

So one of the problems with all the AI stuff is there’s too many tools, too much to experiment with, and too much to try. But, uh, if that— I recommend having a quick look at that and see if it adds any value into the process.

Rikard Edgren [00:31:58]:

Yeah, yeah. Right now I’m, I’m using Cursor. So this is my, my hobby project. So I do normal work at the week, and then on the spare time I do the trouser, and I have my own Cursor license. And the Opus credits I have, I— it’s less than a week and then I’m out of them. So I get new credits at the end of every month. So that’s when I do like the major things or a big refactor or something. I do that and then I do small things with Composer 2.5, waiting for the end of the month.

Rikard Edgren [00:32:32]:

And soon I will take the next leap with Opus.

Alan Richardson [00:32:34]:

Yeah, so open code might help you fill in some of those Yeah. Empty weekends and cause you to not go out in the sunshine anymore.

Rikard Edgren [00:32:41]:

Yeah.

Alan Richardson [00:32:42]:

That would be good. Never see your children and just work on this thing full-time.

Rikard Edgren [00:32:46]:

Yeah, but you know, if you just have— you think in the back of your head, you have like this idea and then the prompt forms without you knowing about it, and then you just write it and then it can work for itself. So I mean, I can still spend time with my children as long as I can paste some prompts.

Alan Richardson [00:33:02]:

Yeah. One of the benefits with— well, one of the disadvantages with Codex is you can connect to it remotely from your phone. So you can tell it to do something, then you can go away, pretend to do something else, but still connect to your AI. So in AI in general, how do you feel about it for software development?

Rikard Edgren [00:33:24]:

In general, like big general, I’m not sure if we are better off with AI than without it right now because it’s used in so many strange ways. So I’m not sure if it’s good for us or not. And I think that applies also for software development because you can use it so many ways. But I also see that it can be extremely powerful. And I guess software development is where it’s at its best because the ones built it are developers. So they built it for themselves and they have used it So I feel it’s a very powerful tool. But you, of course, need— the more power, the more you need to be careful with how you’re using it.

Alan Richardson [00:34:08]:

Yeah, well, the genie is out of the bottle now, so we’re never going to get rid of it.

Rikard Edgren [00:34:13]:

No.

Alan Richardson [00:34:15]:

It’s interesting that, I mean, even you mentioned it in this, where it’s quite expensive to use. So whatever plan you’re on, you only get it a short period of time. But there are so many ways of using it free for smaller activities. So the more experienced you are, the more value you can get out of the free models. And certainly you get most benefit from it with coding because that’s— you’re using it to do so much stuff. But as soon as you have an activity that you can can split down into chunks. Like, say you wanted to summarize a document, and you can split that into chunks of pulling the text out of the Word document. That can be done algorithmically.

Alan Richardson [00:35:00]:

Then you can feed chunks into the AI in smaller parts and then use smaller AI models. So it’s not that AI itself is only good for software development, but certainly AI in Kind of chat-type interface that can review code and work autonomously is excellent there, and it just might need a little bit of support to do everything else.

Rikard Edgren [00:35:22]:

Yeah, and we also need to, to make sure we do the right things. I mean, as, as always, but now even more because it’s so easy to do. It’s so easy to do something that doesn’t really have any value, and you just do it because you can.

Alan Richardson [00:35:38]:

So One thing that helps with that obviously is testing, because you’re checking to see has it done the right thing or has it done something wrong. Um, are you, when you’re trying to use AI, expanding your design skills and modeling skills and architecture skills?

Rikard Edgren [00:35:56]:

No.

Alan Richardson [00:35:57]:

Okay.

Rikard Edgren [00:35:57]:

And, uh, you said in a, in a previous podcast that you, you never use it writing, you never use it for test ID creation because you already have so many test leads. And it’s the same for me. I mean, the problem is not the IDs, the problem is, it’s more of execution time, you have to stop. So I would never use it to create a test strategy or something. It would look good, and someone who doesn’t know the area would think, wow, great, how good it is. But it is not good. And part of the good thing about it is is that if you create it yourself, you know about it. If I have a test idea, we have 20 good, strange test ideas, and then you decide, I will do these 5, then you have done the analysis and it will be better.

Rikard Edgren [00:36:44]:

So, I would rather have the AI— yeah, you can create test ideas or test cases, but you have to run them yourself. I’m not running them for you. And then when it’s interesting, do some testing here and show me the results.

Alan Richardson [00:36:56]:

Okay. So one of the things that has changed slightly from my opinion there is I have been let— I use AI and I tell it to go and find bugs on the site. And I don’t know that I count it as testing. I count it as coverage, exploring, interacting. That’s fine. But it’s— the reason I don’t count it as testing is that there’s no coverage strategy. There’s no thought around everything that we could do. So it can give me some information that sometimes is valid that I can act on, and I’m sure Trowser can do that too.

Alan Richardson [00:37:34]:

You tell it, go away, look at this, find some bugs and exercise it, which for an experienced tester is good because you can let it do that in parallel while you’re off thinking about something else. But The danger is that some people see that, see the result as bugs, and go, all right, now it’s testing, and now I don’t have to worry because it’s doing all the coverage. And when you get the AI to tell you what it has covered, it’s really small in terms of the scope of what it could have done. And so I imagine that there’s a— at the moment, there’s still a role for testers to work out What could we do? What should we be doing? And trying to harness the whole coverage. But AI still can, can be used to find some bugs, and sometimes I’ll let it do that autonomously before I have a look at the system.

Rikard Edgren [00:38:28]:

It’s the same as for me. It’s a complement to my own testing, and I think that together we get the better coverage in total. So it’s, it’s useful, and it’s more attentive to details than me. It’s better than me on security testing. Uh, it can work for more hours without getting bored and stuff like that. So it, it really has value. But I would never do it only the, the AI do the testing, uh, if, if the result is important. And also the, the LLMs are very like, they are pleasers.

Rikard Edgren [00:39:01]:

So if you tell it, um, uh, we have this very nice website, can you check that it’s good? It’s a wonderful website, it has so much value, and you have just small tiny flaws over there. And if you tell it to find as many bugs as possible with high criticality, it will show you, oh, we have 25 super critical stuff, but not all of them are. So one trick I do something is to give the opposite instructions. Like, this is a super good software, try to find some small thing over there. Or tell them, this is a lousy software, there are bugs everywhere.

Alan Richardson [00:39:31]:

Yeah.

Rikard Edgren [00:39:31]:

Tell them about the most important ones, and then you get a better total. And I think, I mean, testing is, testing is in the sampling business. We can never test everything. We can test samples over here and there, and if we look at it with different perspectives, we will get a better whole. And the AI is a new perspective, so it helps us. Yep.

Alan Richardson [00:39:54]:

So have you looked at, tried to look at any of the kind of commercial AI test tools that are out there?

Rikard Edgren [00:40:01]:

I know SmartBear had like a webinar in March, I think, where they were going to showcase something that would change the testing world, the revolution that everybody has been waiting for. And I was so disappointed. I really hoped it would be something great, but it was like test case en masse, like, you know, test case, test case, test case. just test cases everywhere. So, what I’ve seen, I’m not impressed. And I also see that most people are focusing on the automation thing and that part. And I think that handcrafted automation will be better for some period of time. And automation is not all there is.

Rikard Edgren [00:40:47]:

And there’s also a lot of talk about the the pipelines, the CI/CD, and preventing the defects and stuff like that.

Alan Richardson [00:40:56]:

And yeah, it’s interesting because, I mean, tool vendors are in the position where they try and target a niche. And so if it’s a test tool vendor, they’re going to target what they understand of testing, and they understand testing based on the tools they’ve built before, which have all been test case management tools. So consequently, all the AI builds test cases, then automates those test cases and reports on it. But what I find interesting is, um, having tried some of the commercial tools, when it goes away and creates loads of test cases, it doesn’t create loads of test cases. It kind of creates 10 or 20 or 30. And if I look at my code base, I have hundreds, if not thousands, of automated execution paths. And I, I don’t understand. I think one of the things that you said, AI is designed to please.

Alan Richardson [00:41:48]:

It’s also designed to please not use too many tokens. So I think it doesn’t try and do the expanse, so it just tries to curtail it. So you have to keep pushing it to go, all right, give me some more, give me some more.

Rikard Edgren [00:42:01]:

Yeah, but I mean, the solution there is probably you should write your own tool that creates the test cases if you want to have the test cases. We don’t want them, but other people might want them. So, if you know your system, you can use AI to create a tool that creates all your tests. And then that would be cheaper token-wise, and it could be repeatable as well. I mean, I think if you can create a tool that does it instead of asking the AI to do it.

Alan Richardson [00:42:28]:

So, we already have the tool to do that in the sense that if you So for test, just put them in as unit tests, integration tests. They run as part of the code base. They’re automated. That’s what test means at that point. I suspect that the LLM-driven stuff needs to stem from requirements, which the LLM then interprets and then goes, okay, does the system meet this requirement? And then it knows ways of violating the requirement. requirements based on heuristics or something else. One of the things I’m trying to do at the moment with Anyway Data, which is essentially a data generator where you give it a schema and it generates data, is to have an anti-schema. So once you’ve got a schema, it generates data which doesn’t match your schema, and then you can feed that in.

Rikard Edgren [00:43:17]:

Yep.

Alan Richardson [00:43:17]:

That seems like a sensible thing to do, and having something that looks at your requirements and then constantly tries to violate them on an ongoing basis. Because then it doesn’t matter if it doesn’t find anything. But if it does, it’s information. And I think most of the test tools, the AI test tools, are focused on the wrong type of thing. They’re focused on trying to achieve some sort of coverage rather than try and provide you some unique form of information that you would not have thought of or found.

Rikard Edgren [00:43:48]:

Yep.

Alan Richardson [00:43:49]:

But it’s quite hard to justify the cost of that. Yes, we’re paying like $200 for a tool that gives us one thing every week.

Rikard Edgren [00:43:58]:

Yeah. Yeah, but later on, the local models will be runnable on your machine, then the cost will be much cheaper. So that’s where you get the explosion, I think.

Alan Richardson [00:44:09]:

That would be ideal. Unfortunately, no one’s got any graphics cards because they’re all going into data centers. So the local— I mean, at the moment, the local AI could be really good if you could get your hand on the kit. Yeah. So it almost feels like a conspiracy of trying to push us into the cloud-based AI by keeping it off our desktops, even though it could be there. But hopefully it will get there at some point.

Rikard Edgren [00:44:34]:

I came to think about one other realization I had quite recently, uh, how you think about testing. And because I realized since no one likes trouser, it must be I think differently than many other people. And what I realized is that many tool vendors and testers and teams and developers are focusing on testing the code that was recently written. We did this code, we’re going to test this code. My objective is to test the system and test the system that was changed by the code. that was done the last weeks. But I look at the system, not at the code. I mean, for me, code coverage is completely useless.

Rikard Edgren [00:45:16]:

I mean, quite often it’s the code that wasn’t there that was the bug, and you can’t have coverage of that. So, and that’s also what, what Trowser is more. It’s like you look at the system to see what it does. It’s not so concerned with the code because developers are concerned with that.

Alan Richardson [00:45:32]:

Yeah.

Rikard Edgren [00:45:32]:

And I’m bringing a different perspective. And so I think that’s also quite a lot with all these test cases and the coverage, and it’s, it’s more on the code than on the system.

Alan Richardson [00:45:43]:

Yeah, I mean, it’s a different coverage model. Um, when you’re doing proper testing, your coverage model is not code, because that’s, that’s a ridiculous concept that testing would focus on code coverage. But it’s really important to have code coverage to make sure that your code works and doesn’t explode live. But they’re 2 completely different coverage approaches. And that focus of the system is really interesting because the AI finds it quite hard to work out what the system is from the codebase. I think it’s getting better. Like, it doesn’t necessarily know what it’s supposed to do.

Rikard Edgren [00:46:21]:

I’m not sure I agree, Alan. Recently, what I’ve done the last months and stuff like that, typically on a website I would tell it to do a reconnaissance test. Tell me what this site is about, the architecture and stuff like that. And the results is good. It tells me what the site is about. It tells me quite a lot about the architecture and like the APIs and stuff like that. And so I think it has become good.

Alan Richardson [00:46:51]:

So it’s getting better. And I think what I mean is not so much what it can do, what features it exposes, but what the user wants it to be able to do and how they will use it and what the user need is. It’s very good at doing parts of that, but sometimes it gets the wrong nuance for what is there and acceptable. But having something that guides it, like a requirements set or an aims or a goal set or something, I think is also important. But it’s interesting that we haven’t quite got that part yet. But I think that’s hard to sell for people. But I think Paul’s working on some of that. Paul Gerrard is trying to do some sort of requirement-driven coverage type stuff.

Alan Richardson [00:47:35]:

So that, I guess, brings us on to Eurostar. You were at Eurostar a little over a month ago. Because you’re now a legend, because you were doing a Eurostar Legends talk. I think— does that just mean you’re old, or does that mean that you’re a legend?

Rikard Edgren [00:47:52]:

It’s very problematic for me because for many years I tried to have the image of being humble, and it’s getting more and more difficult when you get that label on you.

Alan Richardson [00:48:03]:

Yeah, just the image, not the actual. Okay. So in that talk, you were covering your kind of past experience on testing, which is useful, lessons learned, and the future. So are you able— is that a talk you’re going to repeat, or could you cover some of it without giving away the answers if you ever repeat the talk?

Rikard Edgren [00:48:24]:

I don’t think I would repeat it, but I do like one talk a year, something like that, so not that much. So what it was, was The instruction was to have something more personal about what I’ve done and what I see coming and stuff like that. So I told a bunch of stories from my career. And I can tell you one of the stories now because I don’t want to summarize the speech, but I can tell you one of the stories. My first day as a tester, I had an extremely important test because I was We were testing a Swedish version of Microsoft Internet Explorer 4 Service Pack 2, November ‘98. And we were handed the test cases on paper, back in the days. And I had like a veteran tester with me who was guiding me through the first. So I did the first test case.

Rikard Edgren [00:49:18]:

And then when I was about to do the second test case, Anders told me that in this test case, you should not—

Alan Richardson [00:49:24]:

What?

Rikard Edgren [00:49:24]:

follow the exact steps written there. You should get to the end result, but you should take a different path. Use the keyboard, do something else in between, or use the accelerators. And then I said, yeah, but it says I should do this. Microsoft have told me to execute these tests. Yeah, yeah, yeah, but I can’t do something else? That’s just what they say. They want you to have variation. And it’s easier for them to write test cases this way.

Rikard Edgren [00:49:54]:

So you should do it with variation, and that’s where we will discover more interesting things. And then I said, okay, let’s do it. And that small thing has formed my whole testing career. To do variations, to be observant, to not follow rules strictly, instead follow the intent. and the purpose and discover things about the software.

Alan Richardson [00:50:21]:

Yeah, I mean, it’s a good notion. We can automate the repeated path, but adding variation— you can add some variation into the execution, but the real variation is going to come from the human and adjusting it every single time. And I mean, it’s good that you got that on the first Because I don’t think people learn that until later when something sneaks through the net. So, how do you see the future then for testing?

Rikard Edgren [00:50:56]:

I would say that, I mean, the AI thing now is really changing things, but it’s not changing that much. I mean, you said in a podcast that maybe AI will force the world to rediscover what software testing really involves. Meaning that people have been doing quite bad testing here and there, but now they are forced to do better testing. Because AI can do the weak testing and AI will produce more software that needs testing. And I mean, we have so many things that we know about. I mean, For instance, in some companies I work with quality goals. You have like this list of 15 sentences, what qualities for our product. That’s something good to do then, it’s good now.

Rikard Edgren [00:51:48]:

You can provide that to an AI, to both the developer AI and the tester AI, and they will have a lot of stuff. So my hope is that we have to do more of the good testing that we already knew about, but we have to rediscover it, as you yourself said.

Alan Richardson [00:52:05]:

Cool. So, are you— so, you’re doing Trowser to help you advance your abilities when testing these things. Is there anything that people can do to get better at testing then?

Rikard Edgren [00:52:22]:

I think the first thing to do is to understand what is important about your software. And it’s a lot of things that are important. And to have that in the back of your head all the time and not be too focused on your Jira tickets, that would be the first thing.

Alan Richardson [00:52:39]:

I guess it depends how you write Jira tickets. Of course, if they’re too small and specific.

Rikard Edgren [00:52:43]:

Yeah, but at the same time, if you write this, uh, this super big that involves, and you should be accessibility compliant and it should be easy to use and the words should be clear and simple to understand. No one will do it that way. And yeah, so it’s difficult. And another thing that is important about Krauser is that— I don’t know if you know about Gerd Gigerenzer. He has a tools-to-theories heuristic. And this is general for society. And what he says is that the theories we have are shaped by the tools we are using. So, the tools form our model of the world.

Rikard Edgren [00:53:25]:

So, if you’re using a test tool from a vendor with test cases and everything and a pass/fail, that forms your opinion about what testing is. So, that was my hope with Trowser. Since Trowser is a tool, people use the tool Trowser and then they will think about testing more the way I think they should be thinking about testing.

Alan Richardson [00:53:46]:

So that’s certainly true. If the tool has capabilities and it covers an area that the person knows nothing about, then they have— they either have to ignore that feature or learn it and start using it. And that’s why I like people to open up the dev tools in the browser, because instantly they see all this stuff that they may not know is appropriate for the web or what it actually means and just opens up an entirely new world for people.

Rikard Edgren [00:54:13]:

I met a tester once who said that, and I asked, but why aren’t you using the DevTools? And they said, well, I tried that, but there was so much noise over there. You know, red things and alerts and stuff like that. So, if you aren’t using DevTools, you’re not ready for Trouser either, I would say. Trouser also has DevTools. You get it built in with Chromium.

Alan Richardson [00:54:39]:

Yeah. I think it’s a good— that’s an interesting point to finish on, that concept of augmentation, which then pushes you further. And it can be interactive augmentation like Trowser or the DevTools. Or it can be static augmentation like running axe from the command line to open up a whole new set of information that you’ve never seen before. For and using tools not just to do something but to expand your knowledge about what it’s doing and then build on it and then take that knowledge up to the next level. So I think that’s a useful thing to do. So, uh, so I guess one lesson could be keep adding tools into your process to get new capabilities, as long as you understand them and don’t just lean on it as something to do.

Rikard Edgren [00:55:28]:

Yeah, and also you can— now you can create your own tools. So that would be an encouragement to any tester. Whenever you’re testing something, you feel, oh wow, this is difficult, oh wow, this is boring, uh, maybe you need a tool. Maybe you should create your own tool, because a web extension, you can suck out all the network and find stuff and do regex and stuff, and the AI will do it for you.

Alan Richardson [00:55:54]:

Yep, I think that’s a good idea as well. And, uh, I’ll say that you don’t actually need to pay for AI tooling in order to do it because you can use Open— I should be sponsored by OpenCodeZen. I’m not, but OpenCode, because you use OpenCodeZen, gives you frontier models and you can use them. So you can build tools without actually having to pay for any AI system or run it locally.

Rikard Edgren [00:56:15]:

And also ChatGPT can be used like Python scripts. You want a Python script that looks through these files and find this stuff. Yeah, it can do that, but it cannot do the bigger system with many files involved.

Alan Richardson [00:56:28]:

Cool. All right, so, um, thanks for sharing your knowledge, Rikard. And everyone can find you at thetesteye, where eye is E-Y-E, .com. And there’s a lot of stuff on there to read, and we’ll try and put show notes and things in in there with links. And any last things you wanted to cover or say?

Rikard Edgren [00:56:49]:

No, it was nice talking to you, Adam.

Alan Richardson [00:56:51]:

Well, next time we’ll try and do you on the piano, me on the guitar, and then so we can do that. All right. Thanks, Rikard. Thank you.

Rikard Edgren [00:57:02]:

Bye, everyone. Bye.

Alan Richardson [00:57:04]:

So that was a chat with Rikard Edgren. You can find him at thetesteye.com. And we’ve got all the show notes over at eviltester.com/show.