Express - Elastic Search - Monitoring 3

THE CONTENT ON THIS PAGE WILL BE IMPROVED SOON! TO BE CONTINUED ...

(we are using the elk open source 15.x.x) Once we've created all the files it's time to run the application and see what happens First thing we do, start the docker program on your computer and in the command prompt of our project type `docker compose up`

Dockerfile that sets up a Node.js application, showing steps like setting the base image, copying files, installing dependencies, exposing a port, and defining the start command, with a terminal running docker compose up — Dockerfile Setup for a Node.js Application in Visual Studio Code

The code in the picture is a Dockerfile, which is used to create a Docker image for a Node.js application. Here's a step-by-step explanation of the code:

FROM node:20

This line specifies the base image for the Docker container, which is Node.js version 20.

WORKDIR /src

This sets the working directory inside the container to /app.

COPY package*.json ./

This copies the package.json and package-lock.json files from the host machine to the working directory in the container.

RUN npm install

This runs the npm install command to install the dependencies specified in the package.json file.

COPY . .

This copies all the files from the host machine to the working directory in the container.

EXPOSE 3001

This tells Docker to expose port 3001 on the container. This is the port the Node.js application will use.

CMD ["npm", "start"]

This specifies the command to run when the container starts. It runs npm start, which typically starts the Node.js application.

The terminal shows the command docker compose up being run, which starts all the services defined in a docker-compose.yml file, bringing up the entire environment.

Docker Desktop showing a list of running containers including Elasticsearch, Kibana, and Filebeat, with details on their status, ports, and actions available. — Managing Docker Containers with Docker Desktop

On the pic we see Docker Desktop, which is used to manage Docker containers.
The container list includes several containers, each with information about its name, image, status, CPU and memory usage, ports, and last started time. Containers listed include: elk-monitoring-tutorial, elasticsearch-1, kibana-1, elk-express-ts-monitoring, filebeat

If we click on the 'elasticsearch-1 9200:9200' at the Docker Desktop we should see the pic bellow, but we have to wait a bit.. it is not loading fast..

Browser window displaying JSON data from an Elasticsearch instance, showing cluster and version information including cluster name, UUID, version number, build details, and Lucene version. — Elasticsearch Cluster Information Displayed in Browser

This is a JSON data from an Elasticsearch instance.

By clicking on the 'elk-monitoring-tutorial-elk-express-ts-monitoring 3001:3001' at the Docker Desktop we should see the pic bellow. A simple web page with the text "Hello World!".

Browser window displaying a simple web page with the text `Hello World!` from a local server running on http://localhost:3001. — Basic `Hello World!` Web Page on Localhost Server

The URL bar at the top shows http://localhost:3001, which indicates that the user is accessing a web server running locally on port 3001.

In the browser address if we type `http://localhost:3001/app/home` we should see the pic bellow:

Browser window displaying a web page with the text `Home page was visited` from a local server running on http://localhost:3001/app/home — Custom Route Displaying `Home page was visited` on Localhost Server

The URL bar at the top shows http://localhost:3001/app/home, indicating that the user is accessing a specific route (/app/home) on a web server running locally on port 3001.

In the browser address if we type `http://localhost:3001/app/about` we should see the pic bellow:

Browser window displaying a web page with the text `About page was visited` from a local server running on http://localhost:3001/app/about — Custom About Page Display on Localhost Server

The URL bar at the top shows http://localhost:3001/app/about, indicating that the user is accessing a specific route (/app/about) on a web server running locally on port 3001.

Clicking on Kibana from docker image (http://localhost:5601).. looks like we have to wait a bit...

Browser window displaying the message `Kibana server is not ready yet` from a local Kibana instance running on http://localhost:5601 — Kibana Server Initialization Message on Localhost

The image is displaying a message from Kibana, a part of the Elastic Stack used for data visualization.

In the browser address if we type `http://localhost:3001/app/blog` we should see the pic bellow:

Browser window displaying a web page with the text `Blog page was visited` from a local server running on a Next.js application. — Custom Blog Page Display on Localhost Serve

The URL bar at the top shows http://localhost:3001/app/about, indicating that the user is accessing a specific route (/app/about) on a web server running locally on port 3001.

The result in our visual studio after running `docker compose up` in the command prompt, and we checked the apps in the docker, and they are working, so everything works well!

And it is time to start Kibana by clicking in the docker program again on 'kibana-1 5601:5601'

Kibana interface displaying space selection

and it works! This time Kibana loaded.. it took some time to load. Our next phase is to make the settings in Kibana so we can monitor our application. Clicking on default block, and our next screen should be the pics bellow:

Screenshot of Elastic interface showing the `Add data` page with various options such as ActiveMQ logs, ActiveMQ metrics, Aerospike metrics, Apache logs, and more — Elastic interface displaying the `Add data` page with multiple data integration options

Screenshot of Elastic interface showing the `Add data` page with options such as Audit logs, AWS Cloudwatch logs, AWS metrics, Azure logs, Azure metrics, and more — Elastic interface displaying additional data integration options on the `Add data` page

Screenshot of Elastic interface showing the `Add data` page with various integration options such as AWS Cloudwatch logs, AWS metrics, Azure metrics, Barracuda logs, and more, along with the sidebar menu open to Security and Management options — Elastic interface with `Add data` page and sidebar menu open to Security and Management options

Next step is clicking on the button top left acordion, so we can see the hidden menu, scrolling a bit and then click on the link on the bottom 'Stack Management'

Screenshot of Elastic interface showing the `Welcome to Stack Management 7.15.0` page with the sidebar menu open to Management options — Elastic interface displaying `Welcome to Stack Management 7.15.0` page with Management options in the sidebar

After that click on the 'index management' link:

Screenshot of Elastic interface showing the `Index Management` page with details of the indices, including name, health status, primary and replica counts, document count, and storage size. — Elastic interface displaying `Index Management` page with index details

After that click on the 'Index Pattern'.

Next step is clicking on 'create index pattern' button

Screenshot of Elastic interface with a prompt to create an index pattern, stating `You have data in Elasticsearch. Now, create an index pattern.` and a button to create the index pattern — Elastic interface prompt to create an index pattern for existing data in Elasticsearch

We should create a new index, in our case called: 'elk-express-ts-monitoring' and choosing 'timestamp' option and then click 'Create index pattern' button

Screenshot of Elastic interface showing the `Create index pattern` page with fields for pattern name and timestamp, and a confirmation message indicating that the pattern matches one source — Elastic interface displaying `Create index pattern` page with pattern name and timestamp fields

On this screen again we have to click on the top left menu acordion, and then on 'Discover' link

This sould be the next screen we should see:

On the left menu, the link on the bottom - 'Message':

Right now we shoould see messages from all the visited pages in our application, and if we visit any of those pages again, we should receive new messages here again!

Screenshot of Elastic interface showing the `elk-express-ts-monitoring` index pattern page with a list of fields including timestamp, _id, _index, and more, indicating their type, format, and properties like searchable and aggregatable — Elastic interface displaying `elk-express-ts-monitoring` index pattern page with field details

In our visual studio

Screenshot of Visual Studio Code showing a Dockerfile for a Node.js project with ELK monitoring setup and terminal output displaying logs from the application. — Visual Studio Code with Dockerfile and terminal logs for Node.js ELK monitoring project

This is the end of part 3 - the final part

Click here to continue to the part 1 | Click here to continue to the part 2

ELK Monitoring - https://github.com/cholakovit/elk-monitoring-tutorial