
![Diagram tools](/assets/posts/2025-09-03-diagrams.gif)

Tools:

- https://diagrams.mingrammer.com/
- https://mermaid.js.org
- https://markmap.js.org/
- https://plantuml.com/
- https://github.com/blushft/go-diagrams

#### Source

- [ByteByteGo](https://blog.bytebytego.com/)

Diagram tools


<center>
![System Design](/assets/posts/2025-02-20-system-design.gif)
</center>

Source: [linkedin post](https://www.linkedin.com/posts/goyalshalini_imagine-building-a-system-that-scales-effortlessly-activity-7297140216073445376-GJjT)


The system design universe


Bye [warp](https://www.warp.dev/), hello [ghostty](https://github.com/ghostty-org/ghostty)!

I just need a light and fast terminal with tabs and ability to split panels. Ghostty does it nicely and super easy to config.

I saw it on [Theo's youtube channel](https://www.youtube.com/watch?v=VUxMfyzTM_Y&t=931s)

<center>
<iframe width="560" height="315" src="https://www.youtube.com/embed/VUxMfyzTM_Y?si=lg4i0dVHswjg1NHI" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share" referrerpolicy="strict-origin-when-cross-origin" allowfullscreen></iframe>
</center>

My config so far:

```properties
background-blur-radius = 20
background-opacity = 0.8
copy-on-select = true
font-family = "Geist Mono"
font-feature = -liga
# font-size = 14
quick-terminal-animation-duration = 0
shell-integration = fish
theme = Blue Matrix
unfocused-split-opacity = 0.6
window-padding-color = extend
```


Ghostty


I received a newsletter today, [Which IDEs do software engineers love, and why?](https://blog.pragmaticengineer.com/ide-that-software-engineers-love/).

The top IDE in the list is [Cursor](https://www.cursor.com/). I am playing with it, it seems amazing.

The AI integration with the code is seamless, good feedbacks, fast, good suggestions. And it is a vscode fork, so I can use all my plugins.

I am super impressed, I will play with it for a while and see how it goes.


Cursor


There was a requirement here to stream data from [mongodb](https://www.mongodb.com/) to [Elasticsearch](https://www.elastic.co/), and I decided to research the existing tools.

We could use [elasticsearch-river-mongodb](https://github.com/richardwilly98/elasticsearch-river-mongodb/wiki), [implement a stream process in node](https://medium.com/@amulyakashyap09/stream-mongodb-to-elasticsearch-using-node-js-755b407eb744), some payed solutions... then I found [monstache](https://rwynn.github.io/monstache-site/) - a stream implementation in golang.

## The good parts

- Monstache is just a single binary without dependencies on runtimes like Ruby, Python or PHP - a [scratch image](https://hub.docker.com/_/scratch) should do the work
- It does accept [transformations](https://rwynn.github.io/monstache-site/advanced/#transformation) - working sample [here](https://github.com/adamatti/learn-monstache/commit/51ab57c1be5adb3b60555a74838e49c50ffa05d4)
- [Mappings](https://rwynn.github.io/monstache-site/advanced/#index-mapping) - working sample [here](https://github.com/adamatti/learn-monstache/commit/5ff11d50148354a28c8e027c7935fa0129607013)
- [Versioning](https://rwynn.github.io/monstache-site/advanced/#indexing-metadata)? - working sample [here](https://github.com/adamatti/learn-monstache/commit/9768a64a0e12ac4c0926049b6731c377013df9bc)
- Fetch references? check [embedding documents](https://rwynn.github.io/monstache-site/advanced/#embedding-documents)
- A lot of mongo/elastic [configurations](https://rwynn.github.io/monstache-site/config) - e.g. replay, resume, resume-strategy
- The [docker image](https://hub.docker.com/layers/rwynn/monstache/6.7.17/images/sha256-a275e45f20e2f2e64f30140479469b1241be340cd81ac5641ac391edcb1d092f?context=explore) is less than 15 mb (compressed size)

## Ok, how can I make it faster?

From [the docs](https://rwynn.github.io/monstache-site/advanced/#middleware):

> It is HIGHLY recommended to use a golang plugin in production over a javascript plugin due to performance differences. Currently, golang plugins are orders of magnitude faster than javascript plugins. This is due to concurrency and the need to perform locking on the javascript environment. Javascript plugins are very useful for quickly prototyping a solution, however at some point it is recommended to convert them to golang plugins

For a golang working sample, check [my repo](https://github.com/adamatti/learn-monstache/commit/ae4d813485246f284d4583c4712a4adfaa94d44c)

## The bad parts

It uses [streams](https://x.com/baptistejamin/status/1615310358528376832), and it seems it is not performant as using oplog ([ref](https://jira.mongodb.org/browse/SERVER-46979))

## Conclusion

I would give it a try and perform additional performance tests prior to go to prod.

It is nice it does support [multiple workers](https://rwynn.github.io/monstache-site/advanced/#workers), [High Availability](https://rwynn.github.io/monstache-site/advanced/#high-availability), etc

It has almost 9yrs since [first commit](https://github.com/rwynn/monstache/commit/04c590bf0896330405eeb12614aba1fc8e570fba), [1.3k starts on github](https://github.com/rwynn/monstache), [videos on youtube](https://www.youtube.com/results?search_query=monstache+elastic)

## References

- [Monstache Github Repo](https://github.com/rwynn/monstache)
- [Monstache Website/Documentation](https://rwynn.github.io/monstache-site/)
- [Github - my working sample](https://github.com/adamatti/learn-monstache)
- [A docker compose with all you need](https://github.com/rwynn/monstache-showcase/blob/master/docker-compose.sc.yml)
- [Step by Step how to create a golang plugin](https://github.com/rwynn/monstache/wiki/Go-plugin-guide)


A tool to stream data from mongodb to elasticsearch

Monstache


Since ~2015 - wow, almost 10 years ago - I was using [zsh](https://www.zsh.org/) + [oh-my-zsh](https://ohmyz.sh), and it was great. But after some years it was becoming show, and since I love to try new things I decided to explore new shell options.

TL/DR: I am using [fish](https://fishshell.com/) now. Why?

- It seems faster, at least in those first days / fresh install
- I was using [oh-my-zsh](https://ohmyz.sh) mainly for aliases (e.g. [git](https://github.com/ohmyzsh/ohmyzsh/blob/master/plugins/git/README.md)), but [abbreviations are much better for history and share commands](https://www.sean.sh/log/when-an-alias-should-actually-be-an-abbr/).
- Auto complete seems to works better + syntax highlight
- Functions (super easy syntax)
- All the things that I was using continue to work as expected (e.g. [mise](/blog/posts/2024-09-24-mise), [zoxide](https://github.com/ajeetdsouza/zoxide), etc). No reason to not work, but worth to mention
- Saw others moving from zsh to fish, it seems a natural moving
- [11 reasons here](https://itsfoss.com/fish-shell-features/)

# Super quick start

```bash
# Install fish on OSX
brew install fish

# Add it to possible terminals
code /etc/shells
# Manually add /opt/homebrew/bin/fish (ok, could do it with echo but choose not to)

# Change default terminal
chsh -s /opt/homebrew/bin/fish

# Have fun
```

# Customizations

- I installed [fisher](https://github.com/jorgebucaran/fisher) to add extensions, but didn't find a need for it yet. [Auto pair](https://github.com/jorgebucaran/autopair.fish) is the only one active
  - [fish-abbreviation-tips](https://github.com/gazorby/fish-abbreviation-tips) is great, but since switched aliases for [abbreviations](https://www.sean.sh/log/when-an-alias-should-actually-be-an-abbr/) it doesn't make sense anymore.
- For themes and colors I was using [starship](https://starship.rs/). Decided to stick on it for now, configs [here](https://github.com/adamatti/dotfiles/tree/main/starship).
- [Abbreviations!](https://www.sean.sh/log/when-an-alias-should-actually-be-an-abbr/) You can create it even for parameters and can edit the commands prior to execute.

# Next

- Join [reddit group](https://www.reddit.com/r/fishshell/)
- Take a look on [fisher plugins](https://github.com/jorgebucaran/awsm.fish#readme), [github repos](https://github.com/topics/fish), etc
- Add a star in the [fish-shell](https://github.com/fish-shell/fish-shell) repo


Fish Shell


I saw in twitter some time ago, I think it was suggested by [Akita](https://twitter.com/AkitaOnRails?ref_src=twsrc%5Egoogle%7Ctwcamp%5Eserp%7Ctwgr%5Eauthor) - disclaimer: unable to check here as [twitter still blocked in Brazil](https://www.bbc.com/news/articles/c5y3rnl5qv3o) - about [mise](https://github.com/jdx/mise), a version manager.

Previously I was using [gvm](https://github.com/moovweb/gvm) for golang, [fnm](https://github.com/Schniz/fnm) for node (it is like [nvm](https://github.com/nvm-sh/nvm), but faster), [sdkman](https://sdkman.io/) for java/jvm, some crazy tools for python/ruby, and [brew](https://brew.sh/) for the other (e.g. [bun](https://bun.sh/), [deno](https://deno.com/)).

With [mise](https://github.com/jdx/mise) I can use one tool for all. It is great because:

### 1. It does respect the other format configurations

`.ruby-version`? `.nvmrc`? We don't need to change existing projects to use it.

### 2. It is easy to install

`mise install` in an existing app, `mise use node@lts` / `mise use deno@latest` / etc to add a new dependency.

You can also install global ones, like `mise use --global opentofu@latest`

### 3. Easy to check required versions

Just run `mise ls`

### 4. Easy to upgrade all

`mise up`

### 5. It has hundreds of integrations

Check it all at [here](https://mise.jdx.dev/plugins.html)

### Others

It does integrate with bash / zsh for auto version switch (switch on cd).

It does support environment vars control and task automation, but I do prefer other tools for that.

I hope you enjoy it, let me know your thoughts in the comments.


Mise: a version manager


I was trying to find an alternative to [aws-sso-creds-helper](https://www.npmjs.com/package/aws-sso-creds-helper) since it is archived/deprecated.

The "official" AWS solution is not great either (`aws sso login --profile <profile_name>`).

My coworkers are using [granted](https://docs.commonfate.io/granted/introduction). 

The [installation](https://docs.commonfate.io/granted/getting-started#installing-the-cli) was simple:

```
brew tap common-fate/granted
brew install common-fate/granted/granted
```

The [configuration](https://docs.commonfate.io/granted/usage/automatic-config-generation#all-available-aws-sso-roles) was all auto generated:

```
granted sso generate --sso-region ap-southeast-2 https://example.awsapps.com/start
```

And to use is basically `assume <role>`. I can even add it to my existing scripts.

We can also [export it to .env file](https://docs.commonfate.io/granted/usage/dotenv):

```
assume role-a --env
```

Plus, [we can access different profiles in different browser tabs](https://docs.commonfate.io/granted/usage/console):

```
assume -c <role>
```

Super useful, highly recommended.

# Update - 2025.11.19

Use [this](https://github.com/fwdcloudsec/granted/discussions/897#discussioncomment-14654787) if `assume` does't show.


AWS Granted


Ok, Make/Makefiles are great. I spoke about then in the [last post](/blog/posts/2024-04-08-makefile).

But what if:

- I want to break them in different files
- Use a better syntax (maybe yaml?)
- Cache the tasks, even the ones that don't generate artifacts (e.g. lint)
- put tasks and aliases close
- validate the task execution
- have global tasks to run in any folder (e.g. backup)
- dry run
- watch task

#### Taskfile to rescue

Just need to [install Task](https://taskfile.dev/installation/) (or go-task on [github](https://github.com/go-task/task)).

The [documentation](https://taskfile.dev/) is pretty simple IMHO, but I want to highlight some things that I do use in daily bases that I think it does a great difference.

## Show me the code

For this I will use the sample from [previous post](/blog/posts/2024-04-08-makefile) ([makefile here](https://github.com/adamatti/node-sample/blob/main/Makefile))

```yaml
# yaml-language-server: $schema=https://json.schemastore.org/taskfile.json
---
version: '3'

env:
  DATABASE_URL: postgres://postgres:sample@localhost:5432/sample

includes:
  docker-compose:
    aliases: [dc]
    taskfile: ./Taskfile-docker-compose.yaml
  prisma:
    aliases: [p]
    taskfile: ./Taskfile-prisma.yaml

tasks:
  default:
    desc: Show help
    aliases: [h]
    silent: true
    cmds:
      - task -l --sort alphanumeric

  clean:
    desc: Remove node modules
    aliases: [c]
    cmds:
      - rm -rf node-modules

  install:
    desc: Install dependencies
    aliases: [i]
    run: once
    cmds:
      - npm install
    preconditions:
      - test -f package.json
      - test -f package-lock.json
    sources:
      - package.json
      - package-lock.json
    status:
      - test -d node_modules

  run-only:
    aliases: [ro]
    interactive: true
    ignore_error: true
    dotenv: ['.env']
    cmds:
      - task: install
      - task: prisma:generate
      - npx --yes esno src/index.ts

  run:
    desc: Run the project
    aliases: [r]
    cmds:
      - task: docker-compose:up
      - task: prisma:push
      - task: run-only
```

There are a lot of things going on here, so trying to explain some things:

1. The first line is just for auto complete on IDEs (like vscode, zed)
2. We can define global env vars in the script itself (e.g. `DATABASE_URL`)
3. We can [import](https://taskfile.dev/usage/#including-other-taskfiles) part of the scripts. You just need to have your global ones (e.g. terraform, docker-compose, prisma, git, etc). It does work with remote files.
4. You can expose/hide tasks with [internal flag](https://taskfile.dev/usage/#internal-tasks)

## Now the best part - cache

Pay a close look on the install task.

1. it only runs if `package.json` and `package-lock.json` are present ([programatic checks](https://taskfile.dev/usage/#using-programmatic-checks-to-cancel-the-execution-of-a-task-and-its-dependencies))
2. [Prevent unnecessary work](https://taskfile.dev/usage/#prevent-unnecessary-work): it caches the execution using the `source` attribute.
3. at the end it checks if the `node_modules` folder was created.
4. Even caching based on the `source`, if you remove the `node_modules` folder (check) it will run again.

So, you just need to run your daily tasks (e.g. tests, lints, run local) and the script itself will detect what is pending and run it if needed.

No more crazy issues that you forgot to update a dependency or database change.

## Ok, show me a normal day of work

At the first time, you just need to run `task r` to run the app. It will install dependencies, start database, apply db changes and run the app.

But what happen if someone changes a dependency? No problem. Taskfile will detect and will run `npm install` again.

But if someone just changes the db schema? No problem. It will generate the classes again.

If nothing changes, it will just run the app (ok, the docker part is not optimized, but you got the idea).

If need to stop the app and run again you can go direct to `task ro` (ro = run only).

## Full Project

As always, the full sample is available on [github](https://github.com/adamatti/node-sample).

## Closing thoughts

Now go to the [documentation](https://taskfile.dev/usage/).

Remove commands from `README.md` files and confluence/sharepoint. Also remove it from anemic `package.json` 😛

My global taskfiles are shared [here](https://github.com/adamatti/dotfiles/tree/main/taskfile) for reference.

My backup scripts using rsync are also on taskfiles, but this is a subject for another post.


Taskfile - an alternative for Makefiles


Imagine the follow scenario: you do have a node api with typescript, that uses a database (postgres in this sample) and an orm ([prisma](https://www.prisma.io/)). Every time someone in the team adds a new dependency, you need to run `npm install`. 

Database changes? You need to generate the classes again with `npm run prisma generate`. Oh, you also need to apply the changes with `npm run prisma db push` (note: a real world project would use migrations files, so `npm run prisma migrate dev`).

Then you moved to another service. Then moved back... oh, now need to start database again with `docker-compose up -d`. And so one... 

The steps are documented in a readme file. Or you can get it from `package.json` -> `scripts`, but there is no comments there or command order/dependence. 

What do you do? 

#### Makefile to rescue

So, Make is a build automation tool (thanks [wikipedia](https://en.wikipedia.org/wiki/Make_(software)), and it is available by default in all unix systems.

With that you can:
- create goals (tasks)
- create multi line commands
- use vars
- use conditions
- create aliases for the commands
- map task dependencies
- avoid re-running a task if artifact is already created (e.g. build). "Kind" of a cache (strong quotes here)

#### So, show me a sample!

Sure... check this one (full project [here](https://github.com/adamatti/node-sample)):

```makefile
default:
	@echo "Please specify a target to run"

clean:
	rm -rf node_modules

node_modules:
	npm install

docker-compose-up:
	docker-compose up -d

prisma-db-push:
	npx --yes wait-on tcp:5432 && \
	DATABASE_URL=postgres://postgres:sample@localhost:5432/sample npm run prisma db push

prisma-generate:
	npm run prisma generate

run-only:
	DATABASE_URL=postgres://postgres:sample@localhost:5432/sample \
	npx --yes esno src/index.ts

run: docker-compose-up prisma-generate prisma-db-push run-only

format:
	npm run prisma format
	npx --yes biome format --write ./src

# Aliases
install: node_modules
dcu: docker-compose-up
pdp: prisma-db-push
pg: prisma-generate
r: run
ro: run-only
first: clean install run
fresh: first
f: fresh
```

As you can see, a simple task is just 

```makefile
taskname: dependency1 dependency2
    my list of commands
    second command here
    etc
```

with the sample provided, I can install dependencies with `make node_modules`. 

If you run it again, it will do nothing as the `node_modules` folder is already present (task name = folder name).

If I want to run my app the first time, I just need `make fresh`, and it will:
- install dependencies
- start database
- generate orm classes
- update database
- run the app

If I want to run the app again, I don't need the full steps. I can go direct to `make run-only`.

#### Ok, I am lazy than that

With dependencies, you can create aliases. e.g.

```makefile
r: run
ro: run-only
```

Now I just need `make r` to run my app. Or `make ro` to run-only.
Note: I am lazier than that, I also [have an alias](https://github.com/adamatti/dotfiles/blob/bb23c786a3181b977874efe83530b65ec5de804a/zsh/aliases.zsh#L3) to run `make` with `m` only 😛

#### How about the default task?

It uses the first task. Or the one defined on `.DEFAULT_GOAL := help` (add it as one of the first lines).

#### Ok, but how about documentation

There is a [trick](https://marmelab.com/blog/2016/02/29/auto-documented-makefile.html) for that - just add those lines in the beginning of the file:

```makefile
.PHONY: help

help:
	@grep -E '^[a-zA-Z_-]+:.*?## .*$$' $(MAKEFILE_LIST) | sort | awk 'BEGIN {FS = ":.*?## "}; {printf "\033[36m%-30s\033[0m %s\n", $$1, $$2}'
```

...plus, `## comment` in the tasks you want comments. E.g.

```makefile
clean: ## remove dependencies
	rm -rf node_modules

prisma-generate: ## generate prisma classes
	npm run prisma generate
```

Now you just need to run `make` or `make help` to see all tasks and comments, sorted by task name, e.g.

```shell
> make help
clean                          remove dependencies
docker-compose-up              start dependencies
first                          first execution
format                         format code
fresh                          fresh execution
install                        install dependencies
prisma-db-push                 push prisma schema to database
prisma-generate                generate prisma classes
run-only                       run the project
run                            run the project + dependencies
```

#### Advanced

- How to hide the command from output? Just add `@` in the front of the command.
- call an existing task in the middle/end of my commands? `$(MAKE) your task`
- cache task? the task should match with the output folder or file name

#### Show me the full project

It is [here](https://github.com/adamatti/node-sample), and the makefile is [here](https://github.com/adamatti/node-sample/blob/main/Makefile)

#### Why post about makefile?

Because this is the [most watched video on my channel](https://www.youtube.com/watch?v=h4aV3C6BRDQ). And a lot of developers still adding commands on readme files on partially on package.json. We developers should automate things, even our daily tasks.