If you have encountered this sort of issue
* Starting FTP server vsftpd
* vsftpd failed - probably invalid config.
try this one. Inside /etc/init.d/vsftpd replace these lines
start-stop-daemon --start --background -m ...
n=0
while [ ${n} -le 5 ]
with this one
start-stop-daemon --start --background -m ...
sleep 1 # this one, will remove race condition
n=0
while [ ${n} -le 5 ]
I have spent few hours to resolve this one ;)
That’s yet another post about making live easier through Docker. This time, I am focusing on FTP.
Testing FTP features in PHP based project requires running FTP server. And, let’s be honest, this is a place where you can face some issues. First of all, you have to have FTP server running, you have to convince your FTP server admin to give you some space there, and you have to convince him to allow you to play with WWW storage :) I guess, you can call it a challange.
Another approach is to set up your own machine – doable.
And yet another approach is to use specially crafted Docker image. Brought to you as a dedicated solution for all your NetCAT testing related troubles ;)
> git clone https://github.com/mkowsiak/FTPDocker.git
That’s almost everything you need to do. There are just few more steps to go. Simply build it, run it and let it go.
You can run container with one of three, available, flavors: implicit_ssl, explicit_ssl, or no_ssl.
> docker build -t ftptest .
> docker run -i -t \
-p 80:80 \
-p 2020:20 \
-p 2021:21 \
-p 990:990 \
-p 21100-21110:21100-21110 \
ftptest /bin/start.sh no_ssl
After it is successfully started you will be able to read
*************************************************
https://github.com/mkowsiak/FTPDocker
*************************************************
NetCAT FTP testing server
-------------------------
- FTP User: html
- FTP Password: html
-------------------------
That’s it. We have server running. And, it’s not just a FTP server. We have apache2 on board as well.
To get it running inside NetBeans, we simply have to configure PHP Application from Remote Server. Once you are there, make sure to use proper settings for the project
and location of the server (make sure to clear Upload Directory) field.
While setting up FTP connection, you can choose Pure FTP when you have started container with nossl option, or you can use Explicit FTP using TLS when you have started container with ssl option.
That’s it. You have your own, remote, PHP project ready for development
If you work with remote machines, and you need to take a brief look at some changes in files it might be really painful.
If need a quick glimpse into differences between two files, you can always use diff.
diff file_a.txt file_b.txt
3c3
< It's alive!
---
> And his name will be forty and four.
you can even get some context, together with difference
diff -u file_a.txt file_b.txt
--- file_a.txt 2018-10-26 11:28:40.000000000 +0200
+++ file_b.txt 2018-10-26 11:28:40.000000000 +0200
@@ -1,3 +1,3 @@
Hello world!
-It's alive!
+And his name will be forty and four.
But, let’s be honest, that’s not what regular developers want to see. I know there are people who are extraordinary.
– I wanna tell you my secret now
– OK
– I see dead people
However, for most of us, seeing things as they are, visually, where you can actually see the difference served on the plate in front of you, is a sort of warm place where we feel comforted. And, what’s more important, you can have it, while working with CLI.
First of all, you can always resort to Vim. Yes, old good Vim can provide you with visualization of differences.
> vim -d file_a.txt file_b.txt
Second approach, is to use dedicated tool called: ydiff
All you have to do, is to download it from GitHub.
> curl -ksSL https://raw.github.com/ymattw/ydiff/master/ydiff.py > ~/bin/ydiff
> chmod +x ~/bin/ydiff
and pass outcome of diff directly to ydiff.
> diff -u file_a.txt file_b.txt | ~/bin/ydiff -s
And what is your preferred way of checking differences while working with CLI?
If you want to test remote profiler in NetBeans, it might be quite a struggle. Main source of the problem lays in the fact, you need remote code running. You have few options here: locally started code (you can access it via localhost), Virtual Machine with all the components (JDK, source code, profiler pack), dedicated bare metal based solution (but you need a machine to spare). Anyway, in all cases you have to copy files back and forth, that’s a little struggle. Not to mention time required to setup environment on target machine.
There is, however, yet another way. It simplifies the whole process quite heavily. You can use Docker.
First of all, you need a Docker. But I assume that you either have it somewhere or you know how to install it: Docker.
Then, you need to create Remote profiling pack.
Let’s say, you have saved it here: /tmp/profiler-server-linuxamd64.zip. At the moment, Docker based solution supports only Linux (Intel/AMD) 64bit JVM.
All you have to do, is to build Docker image. Unfortunately,profiler-server-linuxamd64.zip must be at the same level as Dockerfile. You have to pass just one argument – location of JDK you want to install inside Docker container (jdk_location).
> git clone https://github.com/mkowsiak/ProfilerDocker.git
> cd ProfilerDocker
> cp /tmp/profiler-server-linuxamd64.zip ./profiler-server-linuxamd64.zip
> docker build -t profiler \
--build-arg jdk_location=\
"https://download.java.net/java/GA/jdk11/13/GPL/openjdk-11.0.1_linux-x64_bin.tar.gz" \
.
Once you have image created, you can run it: docker run -p 5140:5140 profiler.
As you can see, Java is waiting for a profiler to connect on port 5140. All we have to do is to press Attach inside NetBeans.
That’s it! Profiled code starts to roll, and you can observe results inside NetBeans.
Note 1: In case you are running on macOS make sure to set memory to some reasonable size. Otherwise, your Java code will fail with java.lang.OutOfMemoryError. Take a look here: Docker and memory settings
Note 2: Sometimes, it’s good to clean Docker images so you don’t run out of disk sapce. Take a look here: Cleaning Docker images.
By default, macOS picks up Java version based on highest number that is located here /Library/Java/JavaVirtualMachines. This is quite painful in case you have multiple versions of Java and willing to switch back and forth. You can, of course, export JAVA_HOME like this
> export JAVA_HOME=$(/usr/libexec/java_home -v 11)
> export PATH=${JAVA_HOME}/bin:${PATH}
You can disable Java versions by moving file /Library/Java/JavaVirtualMachines/jdk-##.jdk/Contents/Info.plist to “backup” – Info.plist~. If there is no Info.plist file, this particular version of Java is not taken into account.
Fortunately, in NetBeans, you can pass location of JDK (one that will be used by NetBeans) via --jdkhome argument.
# If you have JDK 11 installed, but you want to run NetBeans
# with version 1.8 simply point to this particular version
# by passing it's location with --jdkhome argument
> ./netbeans --jdkhome \
/Library/Java/JavaVirtualMachines/jdk1.8.0_192.jdk/Contents/Home/
That’s it.
Alternatively, you can modify file etc/netbeans.conf and set value of netbeans_jdkhome=”/path/to/jdk” to location you want to use.
I was struggling with Lightning plugin for quite some time. The reason was that I was getting invitations with mixed up times. Sometimes, they were correct, sometimes, completely crazy.
Eventually, it turned out that Thunderbird has it’s own time zone settings. Who would have guessed?
If you fix this one, all the meetings will start appear with correct date/time settings.
are so painful. The pain comes from the fact that you have to extract MANIFEST.MF to learn what’s inside.
> unzip -qc log4j.jar META-INF/MANIFEST.MF
or
> unzip -p log4j.jar META-INF/MANIFEST.MF
or
> vi log4j.jar
At some point, I have decided to pass size of the log file created by stdroller.
I thought it will be hard as hell to pass SI prefixes. I was all wrong. It’s a piece of cake to parse the value with format like: ./stdroller --limit=10G or ./stdroller --limit=1k, etc.
All you have to do is to use strtoull function. It returns location of first character that is not a part of your positional system (I am using system with base 10). So, you get index of SI prefix for free. Isn’t that cool?
limit = strtoull( optarg, &next_idx, 10 );
if( *next_idx != '\0' ) {
if( *(next_idx + 1) != '\0') {
printf("Ups. You have passed incorrect number\n");
exit(1);
} else {
switch( *next_idx ) {
case 'K':
case 'k':
limit *= pow(10,3);
break;
case 'M':
case 'm':
limit *= pow(10,6);
break;
// you can put additional cases here for G/g, T/t, P/p, etc.
default:
printf("Ups. Incorrect SI prefix\n");
exit(1);
}
}
}
If you need a huge file, and you need it fast
# macOS/linux
dd if=/dev/zero of=bigfile bs=1024 count=1048576
# macOS
mkfile 1G bigfile
# linux
truncate -s 1G bigfile
# linux
fallocate -l 1G bigfile
At the moment, you can’t get nice looking, full-blown installer of NetBeans for macOS, yet. However, running it is fairly easy. Just make sure you have recent JDK installed – just get it from here: JDK 11
Note! Please, make sure you can survive with new licensing model preparred by Oracle. It’s quite different comparing to what it was in the past: Java SE license. It might be more suitable for you to stick to GPL based version of JDK: OpenJDK.
I, personally, decided to use Open JDK. Just download tar.gz, put it inside ~/opt and make sure to export system variables
> mkdir ~/opt
> cd ~/opt
> curl -O https://download.java.net/java/ga/jdk11/openjdk-11_osx-x64_bin.tar.gz
> tar zxf openjdk-11_osx-x64_bin.tar.gz
> export JAVA_HOME=$HOME/opt/jdk-11.jdk/Contents/Home/
> export PATH=${JAVA_HOME}/bin:${PATH}
> java -version
openjdk version "11" 2018-09-25
OpenJDK Runtime Environment 18.9 (build 11+28)
OpenJDK 64-Bit Server VM 18.9 (build 11+28, mixed mode)
# If you want to make sure OpenJDK is visible system wide, make sure
# to copy whole structure here
> cd /Library/Java/JavaVirtualMachines/
> sudo cp -r ~/opt/jdk-11.jdk .
# you can also modify your ~/.profile and add following line there
# export JAVA_HOME=$(/usr/libexec/java_home -v 11)
once you have done it, make sure to download NetBeans nightly build. You can find it here: (all files in zip).
You will get huge zip file and inside, you will have something like this
archive
`-- nbbuild
|-- build
|-- nbms
| |-- apisupport
| |-- ergonomics
| |-- extide
| |-- groovy
| |-- harness
| |-- ide
| |-- java
| |-- javafx
| |-- licenses
| |-- nb
| |-- php
| |-- platform
| |-- profiler
| |-- webcommon
| `-- websvccommon
`-- netbeans
|-- apisupport
|-- bin
|-- cnd
|-- cndext
|-- dlight
|-- ergonomics
|-- etc
|-- extide
|-- groovy
|-- harness
|-- ide
|-- java
|-- javafx
|-- licenses
|-- nb
|-- php
|-- platform
|-- profiler
|-- webcommon
`-- websvccommon
you are interested in this location
> cd archive/nbbuild/netbeans/bin
# make sure to use version you really want to use
> ./netbeans --jdkhome ${JAVA_HOME}
and, that’s it :) You can start working with NetBeans 10.
In case you haven’t heard about NetCAT program, yet, take a look here: NetCAT 10.0. NetCAT is a community acceptance testing initiative targeting NetBeans. If you work with NetBeans a lot, I guess this is the best place to support the team behind the product, and (as well) it’s the best place to get the glimpse of what’s coming with the next release.
If you have never ever worked with CLI (on macOS) I suggest to start with iTerm2: https://www.iterm2.com. I just want you to know, this is my personal view, that iTerm2 is way better comparing to default Terminal – one that you can find inside /Applications/Utilities/Terminal.app.
If you have never ever worked with VIM before, I strongly suggest to go through VIM tutorial. Start terminal application, and simply type vimtutor.
> vimtutor
You will find there lots of basics related to VIM: how to move around in the text, how to quit, how to edit and delete text, etc.
Just one comment from my side. Make sure to get familiar with lesson №1.2
Lesson 1.2: EXITING VIM
otherwise, you might get stuck forever ;)
After going through all the basics, I suggest jumping here: http://www.owsiak.org/mastering-vim-understanding-vims-lesser-known-features-for-more-effective-editing-by-damian-conway/. I am pretty sure, you will find this video very comprehensive. It covers lots of fancy features available in VIM.
If you have developed applications using C, you are probably aware of preprocessor. If you have ever worked with JNI code, you recall preprocessor’s directives for sure. You can find lots of them inside JNI based code generated through javah or javac -h.
/* DO NOT EDIT THIS FILE - it is machine generated */
#include <jni.h>
/* Header for class recipeNo001_HelloWorld */
#ifndef _Included_recipeNo001_HelloWorld
#define _Included_recipeNo001_HelloWorld
#ifdef __cplusplus
extern "C" {
#endif
/*
* Class: recipeNo001_HelloWorld
* Method: displayMessage
* Signature: ()V
*/
JNIEXPORT void JNICALL Java_recipeNo001_HelloWorld_displayMessage
(JNIEnv *, jclass);
#ifdef __cplusplus
}
#endif
#endif
These directives let you change the code before it even gets compiled. This is quite powerful feature, it lets you adapt the code to environment you work with. It’s very useful while working with software, where various, incompatible, APIs must be supported – due to various factors.
You can also maintain different behavior in release and debug mode. You can simply control what gets into final version of the product.
It’s fairly easy to achieve similar (not exactly the same), behavior in Java. Java, by design, doesn’t provide preprocessor’s annotations. To get it working, you have to use some external preprocessing engine. Fortunately, there is one: <#FreeMarker>.
You can easily adapt it to get your Java source code preprocessed before it is being passed to Java compiler. Take a look below, inside jpp repo, to get the glimpse of what’s possible.
Have you tried to organize your songs into play lists? And that’s what you have in your little iPod shuffle?
“-All songs”
“-Play list one”
“-Play list two”
“-Play list three”
“-…”
Man, this was driving me nuts. Fortunately, I found out that I could have “VoiceOver” turned on. After that happened, everything becomes way more simpler.
“-All songs”
“-/bʌɪ wʌns/” (do you know what that could be ;) )
“-Daft Punk”
“-Dumplings”
“-…”
Now, it is so much easier to jump between lists. To get it working, you have to check a little box, down in the settings. It is called – “Enable VoiceOver”
There are different ways to create a log in Java
Recently, I have read quite interesing article regarding logs in Java – Logger to też jest biznes. I guess, it summarize experience of most of the people who work with Java code. I guess, most of us went throuhg different means of logging: System.out, Apache Log4j, Java platform based Logger, Apache Commons Logging, etc. We all have already been there. I bet, everyone has worked with different logger at least once during his/hers career. But it can get even worse.
Native code jumps in
There is even more into the game when you start embed native code calls inside Java code. And, there is a new player to the game, every time you call your Java from a native application (e.g. SOAP based calls from GSOAP based code). These are the places where real headache starts when it comes to Grand Logger Unification.
System.out is not always what you think it is
In general, all stdout streams (in different languages) are a little bit different things. System.out, stdout – in C, unit 6 – in Fortran, sys.stdout – in Python. All of these may behave slightly different and catching and piping data into one, unified logger might be a tricky thing to do. Remember that most of the legacy code, in C/Fortan, will typicaly print messages directly to stdout. You have to handle this situation somehow.
Getting rid of debug code – Apache FreeMarker to the rescue
If you code in C, you are probably familiar with preprocessor commands that can throw parts of the code away from your sources. It means that the code is really not there, in the final binary. This is something different comparing to logging with DEBUG level. In this case, there is no chance to accidentally call part of the code (e.g. by changing logger’s settings). This is just because of the fact, that code is not there. This might be useful in case you want to make sure nobody sees your “@#$%*&” messages – even accidentally.
You can easily simulate this kind of behavior with Apache FreeMarker.
Take a look below
import freemarker.template.*;
import java.util.*;
import java.io.*;
public class Parser {
public static void main(String[] args) throws Exception {
if(args.length != 2) {
System.out.println("You have to specify source file and it's location");
System.out.println("This way, you can easily parse lots of files with find");
System.out.println("");
System.out.println("find src -name \"*.java\" \\");
System.out.println("-exec basename {} \\; \\");
System.out.println("-exec dirname {} \\; \\");
System.out.println("| tr '\\n' ' ' | xargs java -cp .:./freemarker.jar Parser");
System.exit(1);
}
Configuration cfg = new Configuration(Configuration.VERSION_2_3_27);
cfg.setDirectoryForTemplateLoading(new File(args[1]));
Template temp = cfg.getTemplate(args[0]);
Writer out = new OutputStreamWriter(System.out);
Map root = new HashMap();
root.put("debug", new Boolean(true));
temp.process(root, out);
root.put("debug", new Boolean(false));
temp.process(root, out);
}
}
and let’s say, we have src structure like this
src
`-- Test.java
and Test.java file looking like this
public class Test {
// Debugger is set to: ${debug?c}
public static void main(String [] arg) {
/*<#if debug>*/ System.out.println("DEBUG message"); /*</#if>*/
System.out.println("This code is always executed");
}
}
we can easily generate debug/release versions of the file following way
> find src -name "*.java" \
-exec basename {} \; \
-exec dirname {} \; \
| tr '\n' ' ' | xargs java -cp .:./freemarker.jar Parser
It’s worth noting, that source file itself (Test.java) is a valid Java code. It means, we don’t destroy things to get FreeMarker features.
You can also find quite interesting approach to the topic here: Making fun with Java and C preprocessor. This is quite different approach as we end up with broken Java code. Still, if you generate your code automatically, it might be the case you don’t care much whether code compiles or not before it goes through the preprocessor.
And remember, that sometimes, printf is not a bad idea
A few well-chosen test cases and a few print statements in the code may be enough.
Some programs are not handled well by debuggers: multi-process or multi-thread programs, operating systems, and distributed systems must often be debugged by lower-level approaches. In such situations, you’re on your own, without much help besides print statements and your own experience and ability to reason about code.
— The Practice of Programming – Brian W. Kernighan and Rob Pike
Modern Fortran Explained: Incorporating Fortran 2018 by Michael Metcalf and John Reid is already on the way! I am still waiting for it to arrive. It will be some sort of back to the future ;) The oldest “modern” language with new, fancy, features ;)
This time, all in red ;)
Sometimes, you need to go back in time. Recently, I had to develop simple log roller – I wanted to be system independent. Who would have guessed you could find a straightforward solution here ;)
As for the roller itself, you can find it here
I was playing with RStudio recently. And there was this annoying “issue” with plots. I am calling it “issue” as I am not quite sure whether this is a bug or a feature.
The problem was that plots were drawn outside plot area. Axises were missing, title was too high, etc.
I was trying to change plot margins, to squeeze the plot inside plotting area, change the font size, and nothing really worked.
What solved the problem was this tiny line of code
dev.off()
That’s it. All you have to do is to call dev.off(). And then, plot again.
Recently, I was little bit surprised by the output of du command. It turned out that size of a directory can vary, heavily, depending on the filesystem that is used for storing files. I was examining the size of application at different locations and what struck me were the differences in size reports.
# at pfs
> du -s -h directory
16G directory
# at afs
> du -s -h directory
6.3G directory
# at tmpfs
> du -s -h directory
6.4G directory
# how much space does it really take
> tar cf - directory | dd of=/dev/null bs=1M
...
6.7 GB copied, 23.1807 s, 291 MB/s
You can find very comprehensive launchd tutorial here: http://www.launchd.info
