Building OpenCoarrays on macOS – everything from the sources

There are quite a lot of perquisites for this article, but there are two, most important ones. You will need: huge cup of coffee/tea and something to do between various compilation steps (good book would be handy, maybe some movie on Netflix).

There are few steps you have to do, if you want to have everything built from the sources.

1. GCC

This part of tutorial is heavily based on this article – in fact I have stolen the whole idea: https://solarianprogrammer.com/2019/10/12/compiling-gcc-macos/. However, I have changed it a little bit to make all the places local – I want things to be installed inside my $HOME/opt. If you prefer having gcc installed system wide, proceed with the steps above.

First of all, make sure to create place where all the sources and all the compiled codes will go. Note that in case of macOS I am strongly against installing stuff inside /usr. First of all, you have no idea when and what Apple decides to do with central locations – they can protect them, remove, change name, do whatever they want. Second thing is that I prefer to keep all the things close to my $HOME so I can easily move things around and pick arbitrary location for the stuff I build.

> mkdir -p $HOME/opt/usr/local
> mkdir -p $HOME/opt/src

Download gcc.

> cd $HOME/opt/src
> mkdir gcc
> cd gcc
> curl -L -O https://ftpmirror.gnu.org/gcc/gcc-9.2.0/gcc-9.2.0.tar.xz
> tar zxf gcc-9.2.0.tar.xz

Get all the dependencies.

> cd gcc-9.2.0
> contrib/download_prerequisites

Now, this is the step I have stolen from the page mentioned above. We have to create artificial system root. The reason here is that Apple removed /usr/include and moved it inside XCode based platforms. Note that I prefer to create everything inside my $HOME/opt.

> mkdir -p $HOME/opt/usr/local/gcc_system_root
> cd $HOME/opt/usr/local/gcc_system_root
> ln -s /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/Library .
> ln -s /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/System .
> mkdir usr && cd usr
> ln -s /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/bin .
> ln -s /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/lib .
> ln -s /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/libexec .
> ln -s /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/share .
> cp -r /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include include

Now, we have to alter the file Availability.h. Apply this patch (save it inside your $HOME/Availability.h.patch) (you can download it here).

--- /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/Availability.h	2019-11-07 07:07:28.000000000 +0100
+++ Availability.h	2020-01-18 12:17:58.000000000 +0100
@@ -307,6 +307,12 @@
     #define __OSX_AVAILABLE_BUT_DEPRECATED_MSG(_osxIntro, _osxDep, _iosIntro, _iosDep, _msg)
 #endif

+#ifndef __OSX_AVAILABLE_STARTING
+    #define __OSX_AVAILABLE_STARTING(_osx, _ios)
+    #define __OSX_AVAILABLE_BUT_DEPRECATED(_osxIntro, _osxDep, _iosIntro, _iosDep)
+    #define __OSX_AVAILABLE_BUT_DEPRECATED_MSG(_osxIntro, _osxDep, _iosIntro, _iosDep, _msg)
+#endif
+

 #if defined(__has_feature)
   #if __has_feature(attribute_availability_with_message)

apply it following way

> patch -b $HOME/opt/usr/local/gcc_system_root/usr/include/Availability.h $HOME/Availability.h.patch

Now, we can configure the build of gcc.

> cd $HOME/opt/src/gcc/gcc-9.2.0
> mkdir build && cd build
> ../configure --prefix=$HOME/opt/usr/local/gcc-9.2.0 \
               --enable-checking=release \
               --enable-languages=c,c++,fortran \
               --disable-multilib \
               --with-sysroot=$HOME/opt/usr/local/gcc_system_root \
               --program-suffix=-9.2.0

And, once it’s finished, we can build it and install it

> make
> make install

Eventually, you can test it

> cd $HOME
> echo '#include <stdio.h> \nvoid main() { printf("Hello from gcc-9.2.0\\n"); }' \
$HOME/opt/usr/local/gcc-9.2.0/bin/g++-9.2.0 -x c -o hello -; ./hello

If you want, you can update your PATH variable such way it points to this location

export PATH=$HOME/opt/usr/local/gcc-9.2.0/bin:${PATH}

2. MPICH

You need MPICH for OpenCoarrays. Note that MPICH serves as transport layer for OpenCoarrays project (it is said to work with OpenMPI as well).

> mkdir -p $HOME/opt/src/mpich
> cd $HOME/opt/src/mpich
> curl -L -O https://www.mpich.org/static/downloads/3.3.2/mpich-3.3.2.tar.gz
> tar zxf mpich-3.3.1.tar.gz
> cd mpich-3.3.1
> export CC=$HOME/opt/usr/local/gcc-9.2.0/bin/gcc-9.2.0
> export CXX=$HOME/opt/usr/local/gcc-9.2.0/bin/g++-9.2.0
> export FC=$HOME/opt/usr/local/gcc-9.2.0/bin/gfortran-9.2.0
> export FC77=$HOME/opt/usr/local/gcc-9.2.0/bin/gfortran-9.2.0
> ./configure --prefix=$HOME/opt/usr/local/mpich-3.3.1
> make
> make install

Once you have it installed, you can test it with simple code.

program main
  use mpi

  integer error, id, p

  call MPI_Init ( error )
  call MPI_Comm_size ( MPI_COMM_WORLD, p, error )
  call MPI_Comm_rank ( MPI_COMM_WORLD, id, error )
  write (*,*) 'Hello: ', id, '/', p
  call MPI_Finalize ( error )
end

Save it inside $HOME/tmp/hello.f90 and build it.

> cd $HOME/tmp
> $HOME/opt/usr/local/mpich-3.3.1/bin/mpif90 -o hello hello.f90
> $HOME/opt/usr/local/mpich-3.3.1/bin/mpirun -np 4 ./hello
 Hello:            0 /           4
 Hello:            1 /           4
 Hello:            3 /           4
 Hello:            2 /           4

You are done with MPICH. We can now move to CMake.

3. CMake

You need CMake in order to build OpenCoarrays. And, again, I prefer to have it installed my way.

> mkdir -p $HOME/opt/src/cmake
> cd $HOME/opt/src/cmake
> curl -o https://github.com/Kitware/CMake/releases/download/v3.16.1/cmake-3.16.1.tar.gz
> tar zxf cmake-3.16.1.tar.gz
> cd cmake-3.16.1
> ./bootstrap --prefix=$HOME/opt/usr/local/cmake-3.16.1
> make
> make install

You can test whether it works following way

> $HOME/opt/usr/local/cmake-3.16.1/bin/cmake --version
cmake version 3.16.1

CMake suite maintained and supported by Kitware (kitware.com/cmake).

4. OpenCoarrays

Now, let’s install OpenCoarrays.

> mkdir -p $HOME/opt/usr/local/OpenCoarrays
> cd $HOME/opt/usr/local/OpenCoarrays
> curl -L -O https://github.com/sourceryinstitute/OpenCoarrays/releases/download/2.8.0/OpenCoarrays-2.8.0.tar.gz
> tar zxf OpenCoarrays-2.8.0.tar.gz
> cd OpenCoarrays-2.8.0
> mkdir opencoarrays-build
> cd opencoarrays
> $HOME/opt/usr/local/cmake-3.16.1/bin/cmake $HOME/opt/src/OpenCoarrays-2.8.0 \
-DMPI_Fortran_COMPILER=$HOME/opt/usr/local/mpich-3.3.1/bin/mpif90 \
-DMPI_C_COMPILER=$HOME/opt/usr/local/mpich-3.3.1/bin/mpicc \
-DMPIEXEC_EXECUTABLE=$HOME/opt/usr/local/mpich-3.3.1/bin/mpiexec \
-DCMAKE_INSTALL_PREFIX=$HOME/opt/usr/local/OpenCoarrays-2.8.0
> make
> make test
> make install

5. First coarray Fortran code

The time is now :) You can run your very first coarray based code.

program hello_world
implicit none
  write(*,*) 'Hello world from ', &
   this_image() , 'of', num_images()
end program hello_world

Save this file inside $HOME/tmp/hello_world.f90.

> cd $HOME/tmp
> $HOME/opt/usr/local/OpenCoarrays-2.8.0/bin/caf -o hello_world hello_world.f90
> $HOME/opt/usr/local/OpenCoarrays-2.8.0/bin/cafrun -np 2 ./hello_world
 Hello world from            1 of           2
 Hello world from            2 of           2

You did it!! You have macOS based environment for coarray based development in Fortran :)

Passing string as #define and resolving it as string

Sometimes, it’s required to pass string that will be passed as preprocessor variable so we can make some choices during compilation phase. If you try to pass string as preprocessor variable, and will try to use it inside C string literals, it will fail.

#include <stdio.h>

#ifndef VALUE
#error VALUE is not defined
#endif

int main(int argc, char **argv) {

  const char* str = "VALUE";
  printf("%s\n", str);
  return 0;

}

and, of course, I learned that the hard way. It’s simple, string literals are not treated by preprocessor.

> gcc -DVALUE=hi_there -o main ./main.c
> ./main
VALUE

The only way to solve it is to use Stringification.

What you eventually have to do to make it work, is following

#include <stdio.h>

#ifndef VALUE
#error VALUE is not defined
#endif

#define string_value_(x) #x
#define string_value(x) string_value_(x)

int main(int argc, char **argv) {

  const char* str = string_value(VALUE);
  printf("%s\n", str);
  return 0;

}

and you can get what you wanted

> gcc -DVALUE=hi_there -o main ./main.c
> ./main
hi_there

This one will still fail for values with :. E.g.: http://my.address will produce

> gcc -DVALUE=http://my.address -o main ./main.c
> ./main
http:

in order to fix that, we have to change a little bit macro, and the way we are passing variable

#include <stdio.h>

#ifndef VALUE
#error VALUE is not defined
#endif

#define string_value_(x) x
#define string_value(x) string_value_(x)

int main(int argc, char **argv) {

  const char* str = string_value(VALUE);
  printf("%s\n", str);
  return 0;

}

and we have to make sure to pass variable surrounded with \".

> gcc -DVALUE=\"http://my.address\" -o main ./main.c
> ./main
http://my.address

X2Go issue on macOS Catalina – X2Go can’t start XQuartz

If you are running X2Go and you get following error

It means something is wrong with your XQuartz installation. First of all, it might be you haven’t installed XQuartz at all. In that case, take a look here: https://www.xquartz.org/releases/.

Another issue might be related to some dangling connections and in fact, issue is somewhere else than missing XQuartz. If you open Console application, it might be you will find crash report for X11.bin application. Something like this

Process:               X11.bin [1603]
Path:                  /Applications/Utilities/XQuartz.app/Contents/MacOS/X11.bin
Identifier:            org.macosforge.xquartz.X11
Version:               2.7.11 (2.7.112)
Code Type:             X86-64 (Native)
Parent Process:        x2goclient [1599]
Responsible:           x2goclient [1599]
User ID:               501

Date/Time:             2020-01-16 11:15:33.932 +0100
OS Version:            Mac OS X 10.15.2 (19C57)
Report Version:        12
Bridge OS Version:     4.2 (17P2551)
Anonymous UUID:        3B3D6D71-2262-1293-1753-FEEB93B6A0D3


Time Awake Since Boot: 12000 seconds

System Integrity Protection: enabled

Crashed Thread:        5

Exception Type:        EXC_CRASH (SIGABRT)
Exception Codes:       0x0000000000000000, 0x0000000000000000
Exception Note:        EXC_CORPSE_NOTIFY

Application Specific Information:
X.Org X Server 1.18.4 Build Date: 20161025
Cannot establish any listening sockets - Make sure an X server isn't already running
abort() called

It means that you have to clean up X11's lock files and sockets. Simply remove

> sudo rm -rf /tmp/.X11-unix

Some people advice to reinstall different version of XQuartz – e.g.: XQuartz-2.7.7. Well, it will look like reinstalling older version solved your problem, but in fact, you simply applied solution that works by accident. It’s simple, when you install XQuartz it cleans stuff and you get the fresh start.

Embedding JVM inside macOS application bundle using XCode

This time, I am showing you how to embed JVM inside macOS application bundle. In this 5 minutes long video I am guiding you step by step how to create application that will contain JVM. Please note that various JVM releases (depending on vendor) may have different requirements and limitations when it comes to embedding. In this video I am using OpenJDK.

You need no maven to run JUnit 5 tests

Whenever I see presentation, article, tutorial, related to JUnit 5 based tests there is Maven as well. I am not quite a fan of Maven. Anyway, I think it’s worth remembering that for running tests you don’t need it. You don’t have to create complex pom.xml file. You can resort to just few lines of code. All you have to do, is to use make.

Let’s say I have following structure of directory

.
|-- Makefile
`-- src
    |-- main
    |   `-- sample
    |       `-- Main.java
    `-- test
        `-- sample
            `-- MainTest.java

and my files look like this

package sample;

public class Main {
  public int add(int a, int b) {
    return a + b;
  }
}

I want to run test that will make sure it’s valid. I can prepare something like this

package sample;

import static org.junit.jupiter.api.Assertions.assertEquals;
import org.junit.jupiter.api.DisplayName;
import org.junit.jupiter.api.Test;

class MainTest {

  @Test
  @DisplayName("2 + 2 = 4")
  void twoPlusTwoEqualsFour() {
    Main m = new Main();
    assertEquals(4, m.add(2,2), "Something went wrong!");
  }

}

I will also need very simple Makefile

assert = $(if $2,$(if $1,,$(error $2)))

JUnit.ver  = 1.5.2
JUnit.jar  = junit-platform-console-standalone-$(JUnit.ver).jar
Maven.http = http://central.maven.org/maven2/org/junit/platform/junit-platform-console-standalone
JUnit.mvn  = $(Maven.http)/$(JUnit.ver)/$(JUnit.jar)

all: check-variable test

check-variable:
	$(call assert,$(JAVA_HOME),JAVA_HOME is not defined)

dirs:
	-mkdir -p target
	-mkdir -p lib

compile-java: dirs
	$(JAVA_HOME)/bin/javac -d target/classes src/main/java/sample/*.java

junit-download:
	curl -s -z lib/$(JUnit.jar) \
          -o lib/$(JUnit.jar) \
          $(JUnit.mvn)

compile-test: compile-java junit-download
	$(JAVA_HOME)/bin/javac -d target/test-classes \
          -cp lib/$(JUnit.jar):target/classes \
          src/test/java/sample/*.java

test: compile-test
	$(JAVA_HOME)/bin/java -jar lib/$(JUnit.jar) \
           --class-path target/classes:target/test-classes \
           --scan-class-path

clean:
	-rm -rf lib
	-rm -rf target

and then, I can simply run make

> make
...
...
Thanks for using JUnit! Support its development at https://junit.org/sponsoring

╷
├─ JUnit Jupiter ✔
│  └─ MainTest ✔
│     └─ 2 + 2 = 4 ✔
└─ JUnit Vintage ✔

Test run finished after 153 ms
[         3 containers found      ]
[         0 containers skipped    ]
[         3 containers started    ]
[         0 containers aborted    ]
[         3 containers successful ]
[         0 containers failed     ]
[         1 tests found           ]
[         0 tests skipped         ]
[         1 tests started         ]
[         0 tests aborted         ]
[         1 tests successful      ]
[         0 tests failed          ]

You can download sample code here

Google Search Console, site map file and WordPress

If you want to generate site-map of your WordPress based blog it’s simple and hard at the same time.

It’s hard because it’s problematic to find good plugin that can get site-map for you.

It’s simple, because you can do it without any plugin at all.

You will need: search being enabled in your blog, some time to find the oldest search result, small helper script.

First of all, start with pressing enter inside your Search field. This will give you page with search results. Something like this.

Note the address in the address bar. It should resemble something like this: http://www.owsiak.org/?s=.

Now, go ahead and click older posts as long as there are any. It will take some time, but you can see how address bar changes it’s value. It will be something like this: http://www.owsiak.org/page/2/?s, http://www.owsiak.org/page/3/?s, etc. In my case, there are 34 pages in total. So, it means, we have to query the blog 34 times for the search results.

Now, let’s automate it

# -- 8< -- cut here -- script.sh -- cut here -- 8< --

#!/bin/bash

for i in `seq 1  34` ; do
  curl http://www.owsiak.org/page/${i}/?s -o page_${i}
done

# -- 8< -- cut here -- cut here -- cut here -- 8< --

and you can run it. Of course, make sure to adapt the script for your needs.

> chmod +x script.sh
> ./script.sh

You will end up with lots of partial results inside files page_1, …, page_34. These are your search results for your blog.

Now, let’s extract the content. Please note that this part may depend on your blog template. So, make sure to properly extract info from the files.

> cat page_* | grep "<li>" | cut -f2 -d"=" | cut -f2 -d'"' > site-map.txt
> tail site-map.txt | sed 's/^/- /'
- http://www.owsiak.org/redirecting-http-traffic/
- http://www.owsiak.org/mr-robot-some-thoughts-about-bugs/
- http://www.owsiak.org/coding-apprentice-todos/
- http://www.owsiak.org/if-you-have-makefile-that-has-weird-name-but-you-still-want-syntax-highlighting/
- http://www.owsiak.org/makefile-all-you-wanted-to-know-about-variable-but-were-afraid-to-ask/
- http://www.owsiak.org/makefile-know-your-location/
- http://www.owsiak.org/scratch-programming-playground-by-al-sweigart/
- http://www.owsiak.org/two-line-prompt-with-colors-and-gadgets/
- http://www.owsiak.org/mwm-confing-for-people-who-like-minimalism-during-remote-work/
- http://www.owsiak.org/makefile-that-calls-itself-without-hardcoding-file-name/

Once you have your site-map.txt you can upload it to your blog page and then, add it inside Google Search Console.

That’s it.

LEUCHTTURM1917 vs. Moleskine – ultimate, final battle ;)

So, you are thinking which calendar should you pick for a coming year.

In my case, I was always struggling whether I should pick Leuchtturm1917 or whether I should stick to Moleskine. I admit, this review is little bit biassed as I use Moleskine calendars for quite some time already. However, it has some objective approach as well. When it comes to notebooks, I definitely prefer Leuchtturm1917 (especially one with red dots) over Moleskine (blank).

Anyway, I will try to give you the feeling of what to expect when it comes to both calendars. I will try to compare them “section” by “section” – if I can say that.

——————

Size

Leuchtturm1917 (at the bottom) gives you definitely more space to write at comparing to Moleskine. Both are quite similar in size, yet, Leuchtturm1917 is wider.

Note that Moleskine comes with plain covers as well. Alice themed one is my arbitrary choice.

——————