Ubuntu terminal tips and tricks

Note: I wrote this article 4 years ago (2016), and it was never published. I decided to publish it as-is. The ideas below may be a bit outdated but could be still useful.

In my daily life, I use a terminal interface intensively and even for actions which I'm doing quite often, I always discover new ways how to do it even better.
In this article, I will cover tips and tricks which I found useful for common issues what we face as PHP developers.

Which is the bad process that eats all of the resources?

One of the utilities that are available anywhere and always is the top command.
By default, it sorts the output by CPU usage, and the output looks like:

As we can see, it's still not very useful because it only shows that the php is the one that uses 100% of CPU. By adding -c you can see full argument list, top -c:

Now we can see that the php artisan this_is_command_name_that_eats_cpu is what we should optimise.

- Use top -o %MEM -c to sort by memory usage.

How to find something in the log files without writing any regex?

There are many ways how to filter log files, but I think one of the easiest is to use `cat` and `grep` combination. You can combine them and get the desired result easily.

1) You want to find out who accessed /v1/auth endpoint on Mon, 19 Jun:
cat your_access_log_filename.log | grep "19/Jun/2017" | grep "/v1/auth" 

2) List all of the POST requests:
cat your_access_log_filename.log | grep "POST"

3) Filter by specific IP address and the endpoint:
cat your_access_log_filename.log | grep "" | grep "/v1/auth" 

In the examples above, we forward access log file contents to the grep utility that filters lines by any keywords. As you can see, already you can reduce output as much as you would like to by adding a new grep condition.

Note: You can count results by adding the wc -l at the end of your statement. 
- cat your_access_log_filename.log | grep "19/Jun/2017" | grep "/v1/auth" | wc -l
This will show you how many times users accessed /v1/auth on Mon, 19 Jun.

How to find a package version number installed on your machine?

Sometimes it's tricky to get an installed package version number, but the only thing what you know is a package name. One of the examples would be the NewRelic agent. You are aware that you ran apt-get install newrelic-sysmond to install it but when you run  newrelic-sysmond -v it does not show a version number.

In these situations very helpfull could be dpkg --status command:
$ dpkg --status newrelic-sysmond
Package: newrelic-sysmond
Status: install ok installed
Priority: optional
Section: non-free
Installed-Size: 4744
Maintainer: New Relic <https://support.newrelic.com>
Architecture: amd64
Depends: libc6 (>= 2.3.4)
 /etc/newrelic/nrsysmond.cfg afeec443525e4401b44190300f428075
 /etc/default/newrelic-sysmond b0c989ee748c95d28cf2bd92dd675e9d
 /etc/init.d/newrelic-sysmond c0a864f84457f8b47529aa2417d0c8ad
 /etc/logrotate.d/newrelic-sysmond 355f68f1c6d5c16e95987c62d5fd3456
Description: The New Relic agent for server monitoring
Homepage: http://www.newrelic.com/

- dpkg --status only will work in the situations if you installed a package by running apt-get install or dpkg -i.

PHP max_execution_time explained


The crucial part of PHP application maintenance and performance issue mitigation is an understanding of the PHP and server limits (configuration). However, they are not that straightforward as we tend to think. In this article, I will explain and give examples of the PHP max_execution_time configuration limit, which in some situations behaves differently than you would expect. 

From the documentation 

The maximum execution time is not affected by system calls, stream operations etc. Please see the set_time_limit() function for more details.

These days this paragraph is significant because PHP applications mostly use their time performing that type of operations. And it means your script can actually take more time to finish than your configuration limit. In other words, it can take more time than you expect.

The max_execution_time limit in action

The following script changes the max_execution_time to 5 seconds (the default PHP max_execution_time is 30 seconds) and performs the endless loop to fill up the execution time.

As we can see, the script was allowed to run for ~5 seconds until it resulted in the 500 status code triggered by the max_execution_time limit:

In the following examples, I'm going to extend this code.

1) SQL queries, Redis, Memcached or any other TCP connection

The time spent executing SQL queries is not counted. For example, you have a script that has a SQL query, and it's execution takes 10 seconds. It means the script will be allowed to run 15 seconds in case if the max_execution_time is set to 5 seconds.

The following example is the way to test it:

As we can see, the script execution time does not include the SQL query execution time.

The same would apply to any other external TCP calls such as:

  • Memcached
  • MongoDB
  • Redis

2) 3rd party HTTP API calls

I decided to make this as a separate paragraph to clearly show that HTTP request (TCP communication) does not count towards the limit.

3) System calls

The total execution time is 20 seconds because 15 seconds of the system call does not count towards the max_execution_time limit.

4) Filesystem calls

Interaction time (such as fwrite) with the filesystem is not limited by max_execution_time.

5) Sleep methods

As we can see, the sleep or usleep methods are not counted towards the max_execution_time limit.

Closing Note 

I hope this article gave you an idea about the max_execution_time setting nature and what to expect from it. Knowing these aspects, in situations where you would like to limit the PHP execution time, you may probably want to consider using the server or PHP-FPM limits instead of max_execution_time