Connect to Github via Netbeans

Create remote git repository

You need to create new github repository before you can connect local repository to it. Of course you need to have an account on github, but it’s not an issue, it’s free.

2016-11-27-00_21_32-photos

On the remote repository find https address of your project and save it for later.

2016_11_27_00_00_13_photos

Create local git repository

After creating new project in Netbeans you need to initialize local git repository.

Define exceptions – .gitignore file

But before you do that, create “.gitignore” file where you define all exceptions. In the source control we usually save only code files and configurations, but not binary files, IDE specific files etc.  The file must be in the project root folder.

build.xml
/nbproject/
/build/
/web/bower_components/
/web/node_modules/

You can always delete exceptions from “gitignore” file and add files to repository later.

Initialize

At project node, with the right click, select “Versioning” and “Initialize Git repository“.  Initialization procedure add all files in the project, except ignored ones,  to the local repository. Adding a file only mark file for version control. File is not in version control until is committed to.

git_initialize_netbeans

Commit to local repository

Your files need to be committed to local repository. With right click on project node, select “Git” and “Commit”.  This will create local branch in local repository which will you connect to github.

2016-11-26-23_49_55-planitia-no-branchIn the repository browser (/Team/Repository/Repository Browser) you see current state of local git:

2016-11-27-00_11_54-jump-list-for-mozilla-firefox

Pull from remote repository

Because you already commit to remote repository, that’s because of automatically added readme.md and licence files, you must pull those files first and MERGE them to the local repository.  Then you will be able to push all local differences back to remote repository.

2016-11-27-00_27_14-jump-list-for-mozilla-firefoxEnter project URL (https://….), username and password.

2016-11-27-00_28_05-pull-from-remote-repository

Connect branch with local origin branch where files will be synchronized from remote server.

2016-11-27-00_28_25-pull-from-remote-repository And finish. You will merge new files (readme.me and licence) from remote to local git.

Push to remote repository

After you pull new files down or if you didn’t have any to pull, you continue with push.

Select right click on project node and execute Git/Remote/Push.

2016-11-26-23_56_45-jump-list-for-mozilla-firefoxConfigure  URL of your project (https://….), username and password.

2016-11-27-00_04_21-push-to-remote-repository

Select local branch to push.

2016-11-27-00_41_58-push-to-remote-repository

And select local origin branch where remote repository is replicated. 2016-11-27-00_42_08-push-to-remote-repositoryClick Finish and your files will be sent to remote github repository.

Verify remote git repository

If everything went right, you will see all your project files and folders on the github and two new files inside your local project.

2016-11-27-01_11_41-bisaga_planitia_-this-is-another-test-project-to-see-how-new-stuff-working-togetThat’s it ! You are connected to new github repository.

TomEE: Java EE server database configuration

Configure database access

I use Apache TomEE server and therefore I need to configure it for database purpose before first use.

Install database driver

Drop database driver jar file in tomee/lib folder.

Configure datasource

Resources are usually defined in  server configuration file.

Add datasource resource definition in configuration file located in  tomee/conf/tomee.xml.

<?xml version="1.0" encoding="UTF-8"?>
<tomee>  
  <Resource id="jdbc/db" type="javax.sql.DataSource">
    jdbcDriver org.postgresql.Driver
    jdbcUrl jdbc:postgresql://localhost:5432/bsg_taskmgr_db
    userName postgres
    password postgres
  </Resource>
</tomee>

Verify configuration

After server (tomee service) restart, search for your datasource in log (example log file: tomee/logs/catalina.2016-11-26.log). You will find log entry with your resource id there:

Configuring Service(id=jdbc/db, type=Resource, provider-id=Default JDBC Database)

If you restart server from inside netbeans,  just search in output window where log entries are shown.

Inject datasource in java code and use it

To inject instance of datasource where connection is needed you simply add annotation “@Resource” above variable declaration:

@Resource(name = "jdbc/db")
private DataSource ds;

Let’s see whole example with select statement (jOOQ):

package com.bisaga.demo;

import static com.bisaga.demo.generated.tables.Book.BOOK;
import com.bisaga.demo.generated.tables.records.BookRecord;
import java.sql.SQLException;
import javax.annotation.Resource;
import javax.sql.DataSource;
import org.jooq.DSLContext;
import org.jooq.SQLDialect;
import org.jooq.exception.DataAccessException;
import org.jooq.impl.DSL;
/**
 * @author igorb
 */
public class HelloService {

    @Resource(name = "jdbc/db")
    private DataSource ds;    

    public String createHelloMessage(String name) {
        String allBooks = "";
        try {
            DSLContext db = DSL.using(ds, SQLDialect.POSTGRES);
            for(BookRecord b : db.selectFrom(BOOK).fetch() ) {
                allBooks = allBooks.equals("") ? "My books: " : allBooks+";";
                allBooks += b.getTitle();
            }
        } catch (DataAccessException e) {
            System.out.println(e.toString());
        }
        return allBooks;
    }    
}

This example code is called from REST JSON service and result in the browser looks like this:

2016-11-26-16_38_26-localhost_8080_helloworld_webresources_hello

JOOQ – database access layer

Add Java object oriented querying – JOOQ – to project

I love “database first” principle when talking to database. Why would I sacrifice majority of SQL powers, just to talk to database with some ORM technology ? Of course it depend on so many factors, but if you need something very close to database, JOOQ is a way to go.

JOOQ is open source if you use it with open source databases (MySql, Postgres, etc.) !

Configuration and setup

You need to download jooq library and unzip it to some folder (H:\JavaLib\jOOQ-3.8.6).

Code generation config

You need to add jooq config xml file to your project. I added jooq.xml to the root folder.

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<configuration xmlns="http://www.jooq.org/xsd/jooq-codegen-3.8.0.xsd">
  <!-- Configure the database connection here -->
  <jdbc>
    <driver>org.postgresql.Driver</driver>
    <url>jdbc:postgresql://localhost:5432/bsg_taskmgr_db</url>
    <user>postgres</user>
    <password>postgres</password>
  </jdbc>

  <generator>
    <!-- The default code generator. You can override this one, to generate your own code style.
         Supported generators:
         - org.jooq.util.JavaGenerator
         - org.jooq.util.ScalaGenerator
         Defaults to org.jooq.util.JavaGenerator -->
    <name>org.jooq.util.JavaGenerator</name>

    <database>
      <!-- The database type. The format here is:
           org.util.[database].[database]Database -->
      <name>org.jooq.util.postgres.PostgresDatabase</name>

      <!-- The database schema (or in the absence of schema support, in your RDBMS this
           can be the owner, user, database name) to be generated -->
           
      <inputSchema>public</inputSchema>
 		
      <!-- All elements that are generated from your schema
           (A Java regular expression. Use the pipe to separate several expressions)
           Watch out for case-sensitivity. Depending on your database, this might be important! -->
      <includes>.*</includes>

      <!-- All elements that are excluded from your schema
           (A Java regular expression. Use the pipe to separate several expressions).
           Excludes match before includes -->
      <excludes></excludes>
    </database>

    <target>
      <!-- The destination package of your generated classes (within the destination directory) -->
      <packageName>com.bisaga.demo.generated</packageName>

      <!-- The destination directory of your generated classes. Using Maven directory layout here -->
      <directory>./src/java</directory>
      <encoding>UTF-8</encoding>
    </target>
  </generator>
</configuration>

Run code generator

The simplest way is with bash command, this way will always work, even without any special build tool or IDE.

#!/bin/sh
DIR="C:/Programs/jOOQ-3.10.1/jOOQ-lib/"
DRIVER="C:/Programs/drivers/postgresql-42.1.4.jar"
VER="3.10.1"
CP=$DIR"jooq-"$VER".jar"
CP=$DIR"jooq-meta-"$VER".jar;"$CP
CP=$DIR"jooq-codegen-"$VER".jar;"$CP
CP=$DRIVER";"$CP
echo $CP
java -classpath $CP org.jooq.util.GenerationTool jooq.xml

This script is compatible with git-bash on windows. The libraries and folders are of latest version, change to your specific version before run.

“jooq” script is saved in the “bin” folder which is on the path. Now I can run it from everywhere.

Inside Netbeans, open the terminal and type jooq command, you should be in project root folder, where configuration for jooq is.

Generated code

Jooq will generate code for all your tables. You will find new files in packages with name “*.generated.*” :

2016-11-25-17_44_00-start

 

 

 

 

 

 

Flyway – database migration tool

How to setup migration tool for any IDE

There are many ways how to setup migration tool for a specific project, here I will use a command line approach.

As you already know by now, flyway just runs your SQL files with migration DDL commands.  You create new file for each change, incrementally as your database evolve.

We need a folder where we will put files under the project tree.

File names should follow special naming convention for versioned migrations.

Command “fw”

Because I use bash (cygwin) environment inside Netbeans IDE, i will first enable “flyway” tool as a command inside cygwin bash terminal. I create “fw” command file and put it in the “bin” folder which is on the PATH.

#!/bin/sh
/cygdrive/c/Programs/flyway-4.0.3/flyway $@

Netbeans project structure

I have simple hello world java web project. I added a resources, db and migrations folders for data migration files:

├── resources
│   └── db
│       └── migrations
│           ├── V201611222253__Create_book_table.sql
│           └── V201611222338__Add_field_book_table.sql

Configuraton

In root folder, where I will run “fw” command, I added configuration file with the name flyway.conf. This file contain important config settings for flyway to work.

flyway.url=jdbc:postgresql://localhost:5432/bsg_taskmgr_db
flyway.user=postgres
flyway.password=postgres
flyway.schemas=public
flyway.locations=filesystem:./resources/db/migrations

There are many possible settings. You can take prepared flyway.conf file as a base for your configuration file.  Look at conf folder under flyway installation folder, there is sample conf file with comments around every possible configuration key.

Final structure

In simplified view (I omit lot of files here) the structure of a project folder looks like :

.
├── build.xml
├── flyway.conf
├── library.xml
├── build
├── dist
├── nbproject
├── resources
│   └── db
│       └── migrations
│           ├── V201611222253__Create_book_table.sql
│           └── V201611222338__Add_field_book_table.sql
├── src
│   ├── conf
│   │   └── MANIFEST.MF
│   └── java
│       ├── com
│       │   └── bisaga
│       │       └── demo
│       │           ├── generated
│       │           ├── HelloResource.java
│       │           └── HelloService.java
│       └── org
│           └── netbeans
│               └── rest
│                   └── application
│                       └── config
│                           └── ApplicationConfig.java
├── test
└── web
    ├── calc.js
    ├── index.html
    ├── META-INF
    │   └── context.xml
    └── WEB-INF
        ├── beans.xml
        └── web.xml

Migrations SQL files

This two files are simple migration files:

create table book (
    id serial not null,
    title text not null,

    constraint book_pk primary key (id)
);

and second file where we add new field to the table :

alter table book add column author text not null;

Running migrations

There are only few commands. For start, you need only two (info and  migrate). After you write your first migration file, just try out “info” command. You should see pending migration record for your SQL file.

$ fw info
$ fw info

Database: jdbc:postgresql://localhost:5432/bsg_taskmgr_db (PostgreSQL 9.4)

+--------------+-------------------+---------------------+---------+
| Version      | Description       | Installed on        | State   |
+--------------+-------------------+---------------------+---------+
| 201611222253 | Create book table |                     | Pending |
+--------------+-------------------+---------------------+---------+

Now you can migrate first file:

$ fw migrate
$ fw migrate
$ Flyway 4.0.3 by Boxfuse

Database: jdbc:postgresql://localhost:5432/bsg_taskmgr_db (PostgreSQL 9.4)
Successfully validated 1 migration (execution time 00:00.012s)
Creating Metadata table: "public"."schema_version"
Current version of schema "public": << Empty Schema >>
Migrating schema "public" to version 201611222253 - Create book table
Successfully applied 1 migration to schema "public" (execution time 00:01.265s).

After adding next SQL file in “migrations” folder you can again try status of current migrations with “info” comand:

$ fw info
$ Flyway 4.0.3 by Boxfuse

Database: jdbc:postgresql://localhost:5432/bsg_taskmgr_db (PostgreSQL 9.4)

+--------------+----------------------+---------------------+---------+
| Version      | Description          | Installed on        | State   |
+--------------+----------------------+---------------------+---------+
| 201611222253 | Create book table    | 2016-11-22 23:36:10 | Success |
| 201611222338 | Add field book table |                     | Pending |
+--------------+----------------------+---------------------+---------+

You see there is a Pending change from your last migration file. After another “migrate” command, again this migrations will all be marked as “Installed”.

Ok, that’s it ! Migrations are now in full working state !

How to change “current” folder to “project” folder

For successful execution of any flyway command we need to be in proper folder first. The proper folder is folder where flyway.conf file reside. This is project root folder in my example.

Don’t forget to install “TerminalExtras” plugin for Netbeans, then you can just press Alt+. when you staying on the project root node. Change directory (cd) command will be executed inside terminal window and focus will be set to terminal window.

2016-11-23-00_56_35-helloworld-netbeans-ide-8-2

 

 

Bash script

How to write bash script

I am using cygwin on windows and I wish to use bash as my primary scripting tool.

Write script file with the special first line which define interpreter. Put file in a folder accessible  from anywhere (folder must be on the PATH). Change file characteristics to be executable (chmod +x).

#!/bin/sh
/cygdrive/c/Program\ Files\ \(x86\)/Notepad++/notepad++.exe $@ &
$ chmod +x note

This script will call notepad++ editor and open file from passed parameters (test.txt).

$ note test.txt

Unix (LF) – proper line endings

Well, don’t forget (like I was), that cygwin is linux on the windows ! It means , your script files need to be in proper format, as on linux.  If you use console editors as for example “nano”,  it will work automatically, but with notepad++ (windows app), files will be created with windows line endings by default.

2016-11-24-21_30_20-h__home_demo-notepadYou need to change line endings with right click menu … to Unix(LF) , or on the menu /Edit/EOL Conversion/.

2016-11-24-22_16_49-settings Example

#!/bin/sh
myvar1=Hello
myvar2=World
echo $myvar1 $myvar2

2016-11-24-22_25_59-settingsIf you ran demo script and you didn’t get expected “Hello World” but only ” World” maybe, well check your line ending definition.

Want to know more ? Look at this nice little beginners bash tutorial or special hacker web site.