May 27, 2017
I happened across this article on a programming language and IDE called Xojo. I never used it under this name, but I have fond memories of using it when it was called RealBASIC. I learned it shortly after it came out in the late 1990s. I found it very intuitive and it was super easy to create fairly nice looking GUI applications for the Mac. In fact, the first optimization code I ever wrote was in RealBASIC, for solving the traveling salesman problem.
In the mid-2000's when I was a grad student, I used it to build a GUI interface to JPL's BlackJack GPS receiver. This included reading data from the receiver, as well as sending commands to it. There were numerous floating windows that displayed data and plots (see screenshot at right). At the time, JPL's interface code only ran on pre-MacOS X systems, which were rapidly becoming obsolete. My RealBASIC app ran great on MacOS X. In fact, I believe JPL requested a copy when they saw it (I have no idea if it lives on somewhere at JPL).
In 2006, I traveled to Germany with a blue clamshell iBook to deliver some updated software to the TerraSAR-X satellite, which included the same type of GPS receiver. I remember during the update process, something unexpected occurred (the details escape me now). In any event, it was required to generate some binary command files to upload to the system in order to set things right. My RealBASIC interface app was already set up to do that, so I fired up my iBook and was able to generate the files in short order. I distinctly remember some surprise from the German engineers that I could do such a thing with this silly looking machine! After uploading the files, things got back on track. TerraSAR-X has been in orbit since June 2007.
I haven't used it in years, but I'm glad to know that RealBASIC is still around.
See also
May 19, 2017
It appears that Flang, a new open source Fortran front-end for LLVM, has appeared on GitHub recently with little fanfare. This is apparently the result of NVIDIA's previously-announced plan to open source the PGI Fortran compiler. Unfortunately, they decided to give it the same name as another earlier attempt to create a Fortran/LLVM compiler (more confusion for poor Fortran programmers). I don't really know how it compares to Gfortran or Intel (PGI appears to be lagging behind on support for the Fortran 2008 standard). Initial tests by Usenet denizens (yes Usenet still exists) indicate that maybe Flang isn't quite ready for prime time. Hopefully it will improve with time. I think it's great news to potentially have another free Fortran compiler available.
See also
May 07, 2017
String and file manipulation in Fortran isn't as bad as you've heard (OK, it's bad, but it's getting better). Sure, modern Fortran only provides the bare minimum of features for this sort of thing out of the box, but various libraries are starting to appear that the modern Fortran programmer can use to deal with a range of different file formats including JSON, INI, CSV, XML, etc. The Fortran user community has been late to the game on this opensource thing. But it does exist and I encourage all Fortraners to use it. You don't have to roll your own file format or write your own parser. If the existing libraries don't have what you need, then help to improve them.
The following is a look at a few different options for configuration files in your Fortran program. It is limited to text files. For binary files, I would highly recommend a standard cross-platform format (such as HDF5). For goodness' sake, do not invent your own binary file format.
Namelists
Fortran namelists are very old-school, but are still part of the standard (indeed this is the only high-level file read/write feature that is built into Fortran). Namelist have a few advantages, but they are generally outweighed by their many disadvantages. Some advantages include:
- It is super easy to use. The reading and writing of the file is automatically handled by the compiler. A namelist is declared like so:
integer :: a
real :: b
character(len=5) :: c
integer :: iunit,istat
namelist /blah/ a,b,c
!set values:
a = 1
b = 2.0
c = 'three'
and can be written by:
write(unit=iunit, nml=blah, iostat=istat)
Which produces a file like this:
&BLAH
A= 1,
B= 2.00000000 ,
C="three",
/
The file can be read by:
read(unit=iunit, nml=blah, iostat=istat)
- It automatically handles any Fortran variables you throw at it (including multidimensional arrays and derived data types).
- Variables in the file can be optional. Any variable not present in the file retains the value it had before the namelist was read.
Disadvantages include:
- Different compilers can and will output slightly different formats, and you have absolutely no control over that.
- If you want to generate or manipulate your input files using a different language, you'll probably have to write your own parser to do it. Hardly any other programming language will have a decent namelist parser or writer available. A notable exception is f90nml, a great Python library for reading and writing namelists.
- The variables in the file must exactly correspond (in name, type, and size) to variables in the code. This turn out to be very restrictive for a lot of reasons. For one thing, it is quite annoying for strings, since it requires you to specify a maximum string length in the variable declaration.
- It can be difficult to diagnose errors when reading the file (e.g, a syntax error, an unrecognized variable, or an array size or string length larger than the Fortran variable). The compiler will usually just throw an I/O error if anything goes wrong. However, it is possible on some compilers to retrieve the line where the error occurred by rewinding one line and then reading it with a normal READ statement.
- It is difficult to tell if a variable is actually present in the file (if one isn't there, then the corresponding variable in the code will just retain whatever value it had before the read was attempted. Thus, the only real way to check if a variable was present or not is to initialize them all to unusual values, and then check for this after the read.)
In my view, there is little reason to start using namelists at this point, since there are now better options available. Unless you have a legacy code that is using them which can't be changed, I would recommend using something else.
JSON
JSON stands for JavaScript Object Notation, and is a very popular and lightweight data-interchange format. There is an interface for practically all known programming languages, including modern Fortran. My JSON-Fortran library was built on an older Fortran 95 library, which I forked in order to be able to use some of the newer Fortran 2003/2008 features. With JSON-Fortran, the example above would be:
type(json_file) :: json
call json%add('blah.a',1)
call json%add('blah.b',2.0)
call json%add('blah.c','three')
Which produces the JSON file:
{
"blah": {
"a": 1,
"b": 2.0,
"c": "three"
}
}
Note that f90nml can also be used to transition away from namelists by converting them to JSON using Python. To go from a namelist to JSON, all you have to do is this:
import f90nml
import json
n = f90nml.read('my_namelist.nml')
with open('my_namelist.json', 'w') as outfile:
json.dump(n, outfile, sort_keys = True, indent = 4)
Using my JSON-Fortran library, you can even convert a JSON file back into a Namelist if that sort of thing is your bag (as shown in a previous post). One disadvantage of JSON is that the standard does not allow for comments in the file. However, some parsers include support for comments (including JSON-Fortran).
INI
INI is a simple, easy-to-write format. Although not really standardized, it is based on the old MS-DOS initialization files. For modern Fortran, the FiNeR library provides a Fortran interface. The INI version of the above file would be:
Here's an example of converting an INI file to a JSON file (using FiNeR and JSON-Fortran):
subroutine ini2json(ini_filename, json)
use json_module
use finer
implicit none
character(len=*),intent(in) :: ini_filename
type(json_file),intent(out) :: json
type(file_ini) :: ini
integer :: ierr
integer :: i
character(len=:),dimension(:),allocatable :: secs
character(len=:),dimension(:),allocatable :: opts
call ini%load(filename=ini_filename,error=ierr)
call ini%get_sections_list(secs)
do i=1,size(secs)
do
if (.not. ini%loop(trim(secs(i)),opts)) exit
call json%add(trim(secs(i))//'.'&
trim(opts(1)),trim(opts(2)))
end do
end do
call ini%free()
end subroutine ini2json
You could call this routine like so:
program main
use ini2json_module
use json_module
implicit none
type(json_file) :: json
call ini2json('test.ini',json)
call json%print_file()
call json%destroy()
end program main
And it would produce the following result for the file above:
{
"blah": {
"a": "1",
"b": "2.0",
"c": "three"
}
}
Note that, in this example, all the variables are returned as strings (getting the numerical values as numbers is left as an exercise to the reader).
Others
- XML (Extensible Markup Language) -- I never really liked this one because it is not easily human readable or editable. There are some Fortran XML parsers out there, though.
- YAML (YAML Ain't Markup Language) -- I'm not as familiar with this one (also I hate recursive acronyms). There is a Fortran interface to it called fortran-yaml, but I haven't tried it.
- TOML (Tom's Obvious, Minimal Language) - I'm not aware of a Fortran parser for this one. This one was created by one of the founders of GitHub.
- Lua - Another interesting possibility is for your configuration file to be written in a full-up scripting language. There are a couple of Lua-Fortran interfaces out there (e.g., AOTUS).
See also
May 02, 2017
GCC (which includes GFortran) version 7.1 has just been released. Various features from Fortran 2003-2015 have been added, including:
- User-defined derived-type input/output (UDTIO).
- Partial support for derived type coarrays with allocatable and pointer components.
- Non-constant stop codes and error stop codes.
- Derived types with allocatable components of recursive type.
- Intrinsic assignment to polymorphic variables.
Partial support for OpenMP 4.5 is also included. It looks like they also added support for several non-standard extensions that are present in legacy code. GFortran still hasn't quite made it to full Fortran 2003 support, over a decade after that standard was published. But they are getting there. Thanks to the small number of volunteers who work on this. You are the real heroes!
See also
Apr 08, 2017
JSON-Fortran 5.3.0 is out. This release includes a few minor bug fixes and a couple of new features. For one thing, it is now possible to easily build or modify a JSON structure without using pointer variables. Here's an example:
program test
use json_module, rk=>json_rk
implicit none
type(json_file) :: f
call f%initialize()
call f%add('name', 'test')
call f%add('inputs.t', 0.0_rk)
call f%add('inputs.x', [1.0_rk,2.0_rk,3.0_rk])
call f%add('outputs.t', 10.0_rk)
call f%add('outputs.x(1)', 11.0_rk)
call f%add('outputs.x(2)', 22.0_rk)
call f%add('outputs.x(3)', 33.0_rk)
call f%print_file()
call f%destroy()
end program test
As shown, the structure can be built by specifying the variable paths. This way, objects, vectors, vector elements, etc. can all be added. The result of this example is:
{
"name": "test",
"inputs": {
"t": 0.0E+0,
"x": [
0.1E+1,
0.2E+1,
0.3E+1
]
},
"outputs": {
"t": 0.1E+2,
"x": [
0.11E+2,
0.22E+2,
0.33E+3
]
}
}
So now, in addition to the previous methods of building a JSON structure (reading it from a file, reading it from a string, or building it yourself using pointer variables), there is now another way. Each method has its uses for different applications. Building it from the path strings can be very convenient and also opens up the possibility of easy conversion between a Fortran namelist and a JSON file. I'll show an example of this in a future post. In a previous post, I showed how to do the reverse (JSON to namelist).
JSON-Fortran is my most popular GitHub project (70 stars as of today). It usually makes it to GitHub's monthly "trending" Fortran projects page. I'm always interested to hear about what people are using it for. I added a page on the wiki to list some of the projects that use it, so if you're using it, feel free to add your project to the list.
Mar 11, 2017
Binary search is a search algorithm that finds the position of a target value within a sorted array. The following binary search routine is taken from NASTRAN, a venerable finite element analysis program from NASA:
SUBROUTINE BISLOC (*,ID,ARR,LEN,KN,JLOC)
C-----
C BINARY SEARCH - LOCATE KEY WORD 'ID' IN ARRAY 'ARR', 1ST ENTRY
C IF FOUND, 'JLOC' IS THE MATCHED POSITION IN 'ARR'
C IF NOT FOUND, NON-STANDARD RETURN
C I.E.
C ID = KEY WORD TO MATCH IN ARR. MATCH AGAINST 1ST COL OF ARR
C ARR = ARRAY TO SEARCH. ARR(ROW,COL)
C LEN = LENGTH OF EACH ENTRY IN ARRAY. LEN=ROW
C KN = NUMBER OF ENTRIES IN THE ARR. KN =COL
C JLOC= POINTER RETURNED - FIRST WORD OF ENTRY. MATCHED ROW
C-----
C
INTEGER ARR(1)
DATA ISWTCH / 16 /
C
JJ = LEN - 1
IF (KN .LT. ISWTCH) GO TO 120
KLO = 1
KHI = KN
10 K = (KLO+KHI+1)/2
20 J = K*LEN - JJ
IF (ID-ARR(J)) 30,90,40
30 KHI = K
GO TO 50
40 KLO = K
50 IF (KHI-KLO -1) 100,60,10
60 IF (K .EQ. KLO) GO TO 70
K = KLO
GO TO 80
70 K = KHI
80 KLO = KHI
GO TO 20
90 JLOC = J
RETURN
100 JLOC = KHI*LEN - JJ
J = KN *LEN - JJ
IF (ID .GT.ARR(J)) JLOC = JLOC + LEN
110 RETURN 1
C
C SEQUENTIAL SEARCH MORE EFFICIENT
C
120 KHI = KN*LEN - JJ
DO 130 J = 1,KHI,LEN
IF (ARR(J)-ID) 130,90,140
130 CONTINUE
JLOC = KHI + LEN
GO TO 110
140 JLOC = J
GO TO 110
END
As you can see, it is a scene of FORTRAN 66 horrors! However, it can be refactored into a nice modern Fortran routine like so:
pure function bisloc(id,arr) result(jloc)
!! binary search of a sorted array.
implicit none
integer,intent(in) :: id
!! key word to match in `arr`
integer,dimension(:),intent(in) :: arr
!! array to search (it is
!! assumed to be sorted)
integer :: jloc
!! the first matched index in 'arr'
!! (if not found, 0 is returned)
integer :: j,k,khi,klo,n
integer,parameter :: iswtch = 16
n = size(arr)
jloc = 0
if ( n<iswtch ) then
! sequential search more efficient
do j = 1 , n
if ( arr(j)==id ) then
jloc = j
return
else if (arr(j)>id) then
return ! error
end if
end do
return ! error
else
klo = 1
khi = n
k = (klo+khi+1)/2
do
j = k
if ( id<arr(j) ) then
khi = k
else if ( id==arr(j) ) then
jloc = j
return
else
klo = k
end if
if ( khi-klo<1 ) then
return ! error
else if ( khi-klo==1 ) then
if ( k==klo ) then
k = khi
else
k = klo
end if
klo = khi
else
k = (klo+khi+1)/2
end if
end do
end if
end function bisloc
I did change a couple of things:
- The original routine was using an obsolete alternate return (note the
*
argument) for an error condition. The new one just returns 0 if there is an error.
- The original routine was passing in the array as a vector and treating it like a matrix (it's either searching a row or column of this matrix, I admit I had a hard time telling). The new version assumes the input is a rank-1 array. You can pass in whatever matrix slices you want (say a column:
a(:,1)
or a row: a(1,:)
).
- I also made it a pure function, because that's how I roll.
Note that NASTRAN was released under the NASA Open Source Agreement version 1.3, a somewhat unusual open source license. It isn't considered a "free" license by the FSF since it includes a statement requiring changes to be your "original creation".
Unfortunately, the trend at NASA (and elsewhere) is to declare old Fortran code like this to be totally unmaintainable. So, instead of refactoring them into modern Fortran standards, they are thrown away and rewritten from scratch using a more "modern" language like C++ (whose roots go back to 1972!).
See also
Mar 05, 2017
I just released some updates to my two most popular Fortran libraries on GitHub: JSON-Fortran and bspline-fortran. Coincidently, both are now at v5.2.0. Details of the updates are:
JSON-Fortran
There are several new features in this release. The biggest update is that now the code can parse JSON files that include comments (it just ignores them). You can even specify the character that identifies a comment. For example, if using #
as a comment character, the following file can now be parsed just fine:
{
"t": 0.12345, # this is the time
"x": 123.7173 # this is the state
}
Technically, comments are not part of the JSON standard. Douglas Crockford, the creator of JSON, had his reasons [1] for not including them, which I admit I don't understand (something about parsing directives and interoperability?) I mainly use JSON for configuration files, where it is nice to have comments. Crockford's suggestion for this use case is to pipe your commented JSON files through something called JSMin before parsing, a solution which seems somewhat ridiculous for Fortran users. So, never fear, now we can have comments in our JSON files and continue not using JavaScript for anything.
Another big change is the addition of support for the RFC 6901 "JSON Pointer" path specification [2]. This can be used to retrieve data from a JSON structure using its path. Formerly, JSON-Fortran used a simple path specification syntax, which broke down if the keys contained special characters such as (
or )
. The new way works for all keys.
Bspline-Fortran
A minor update to Bspline-Fortran is the addition of an extrapolation mode. Formerly, if an interpolation was requested outside the bounds of the data, an error was returned. Now, the user has the option of enabling extrapolation.
See also
- D. Crockford, "Comments in JSON", Apr 30, 2012 [Google Plus]
- JavaScript Object Notation (JSON) Pointer, RFC 6901, April 2013 [IETF]
Feb 12, 2017
The lowly CSV (comma-separated values) file never gets any respect. Sure, it isn't really standardized, but it is a very useful text file format for columns of data, and is frequently encountered in science/engineering fields. Of course, there is no good modern Fortran library for reading and writing CSV files (see [1-2] for some older Fortran 90 code for writing only). Searching the internet (e.g., [3-4]) you will find various ad hoc suggestions for handling these files. It's amazing how little Fortran users expect their language to support non-numerical algorithms. No Fortran programmer would ever suggest that someone roll their own matrix inverse routine, but they don't see anything wrong with having to write their own data file parser (if you told a Python user they had to write their own CSV parser, you'd be laughed out of the place).
So, I created fortran-csv-module (now available on GitHub), a modern Fortran library for reading and writing CSV files. Everything is handled by an object-oriented csv_file
class. Here is an example for writing a file:
program csv_write_test
use csv_module
use iso_fortran_env, only: wp => real64
implicit none
type(csv_file) :: f
logical :: status_ok
! open the file
call f%open('test.csv',n_cols=4,status_ok=status_ok)
! add header
call f%add(['x','y','z','t'])
call f%next_row()
! add some data:
call f%add([1.0_wp,2.0_wp,3.0_wp],real_fmt='(F5.3)')
call f%add(.true.)
call f%next_row()
call f%add([4.0_wp,5.0_wp,6.0_wp],real_fmt='(F5.3)')
call f%add(.false.)
call f%next_row()
! finished
call f%close(status_ok)
end program csv_write_test
Which produces the following file:
x,y,z,t 1.000,2.000,3.000,T 4.000,5.000,6.000,F
Real, integer, logical, or character data can be added as scalars, vectors, and matrices.
When reading a CSV file, the data is stored internally in the class as allocatable character strings, which can be retrieved as real, integer, logical or character vectors as necessary. For example, to get the x
, y
, z
, and t
vectors from the previously-generated file:
program csv_read_test
use csv_module
use iso_fortran_env, only: wp => real64
implicit none
type(csv_file) :: f
character(len=30),dimension(:),allocatable :: header
real(wp),dimension(:),allocatable :: x,y,z
logical,dimension(:),allocatable :: t
logical :: status_ok
integer,dimension(:),allocatable :: itypes
! read the file
call f%read('test.csv',header_row=1,status_ok=status_ok)
! get the header and type info
call f%get_header(header,status_ok)
call f%variable_types(itypes,status_ok)
! get some data
call f%get(1,x,status_ok)
call f%get(2,y,status_ok)
call f%get(3,z,status_ok)
call f%get(4,t,status_ok)
! destroy the file
call f%destroy()
end program csv_read_test
Various options are user-selectable for specifying the format (e.g., changing the quote or delimiter characters). You can choose to enclose strings (or all fields) in quotes or not. The library works pretty well, and there are probably additional improvements that could be made. For one thing, it doesn't properly handle the case of a string that contains the delimiter character (I'll eventually fix this). If anybody has any other improvements, fork it and send me a pull request. The license is BSD, so you can use it for whatever you want.
See also
- J. Burkardt, CSV_IO -- Read and Write Comma Separated Values (CSV) Files. [note: it doesn't actually contain any CSV reading capability]
- A. Markus, FLIBS -- Includes a csv_file module for writing CSV files.
- How to read a general csv file [Intel Fortran Forum]
- Read data from a .csv file in fortran [Stack Overflow]
Oct 29, 2016
For a number of years I have been familiar with the observation that the quality of programmers is a decreasing function of the density of go to statements in the programs they produce. -- Edsger W. Dijkstra
One of the classics of computer science is Edsger Dijkstra's "Go To Statement Considered Harmful", written in 1968. This missive argued that the GOTO statement (present in several languages at the time, including Fortran) was too primitive for high-level programming languages, and should be avoided.
Most people now agree with this, although some even today think that GOTOs are fine under some circumstances. They are present in C and C++, and are apparently used extensively in the Linux kernel. A recent study of C code in GitHub concluded that GOTO use is not that bad and is mainly used for error-handling and cleanup tasks, for situations where there are no better alternatives in C. However, C being a low-level language, we should not expect much from it. For modern Fortran users however, there are better ways to do these things. For example, unlike in C, breaking out of multiple loops is possible without GOTOs by using named DO loops with an EXIT statement like so:
a_loop : do a=1,n
b_loop: do b=1,m
!do some stuff ...
if (done) exit a_loop ! break out of the outer loop
!...
end do b_loop
end do a_loop
In old-school Fortran (or C) this would be something like this:
do a=1,n
do b=1,m
! do some stuff ...
if (done) goto 10 ! break out of the outer loop
! ...
end do
end do
10 continue
Of course, these two simple examples are both functionally equivalent, but the first one uses a much more structured approach. It's also a lot easier to follow what is going on. Once a line number is declared, there's nothing to stop a GOTO statement from anywhere in the code from jumping there (see spaghetti code). In my view, it's best to avoid this possibility entirely. In modern Fortran, DO loops (with CYCLE and EXIT), SELECT CASE statements, and other language constructs have obviated the need for GOTO for quite some time. Fortran 2008 added the BLOCK construct which was probably the final nail in the GOTO coffin, since it allows for the most common use cases (exception handing and cleanup) to be easily done without GOTOs. For example, in this code snippet, the main algorithm is contained within a BLOCK, and the exception handling code is outside:
main: block
! do some stuff ...
if (error) exit main ! if there is a problem anywhere in this block,
! then exit to the exception handling code.
! ...
return ! if everything is OK, then return
end block main
! exception handling code here
The cleanup case is similar (which is code that is always called):
main: block
! do some stuff ...
if (need_to_cleanup) exit main ! for cleanup
! ...
end block main
! cleanup code here
I don't believe any of the new programming languages that have cropped up in the past couple of decades has included a GOTO statement (although someone did create a goto statement for Python as an April Fool's joke in 2004). Of course, the presence of GOTO's doesn't mean the programmer is bad or that the code isn't going to work well. There is a ton of legacy Fortran 77 code out there that is rock solid, but unfortunately littered with GOTOs. An example is the DIVA integrator from the JPL MATH77 library (the screenshot above is from this code). First written in 1987, it is code of the highest quality, and has been used for decades in many spacecraft applications. However, it is also spaghetti code of the highest order, and seems like it would be unimaginably hard to maintain or modify at this point.
Source: XKCD
See also
- E. W. Dijkstra, "Go To Statement Considered Harmful", Communications of the ACM, Vol. 11, No. 3, March 1968, pp. 147-148 [online version here]
- E. A. Meyer, "Considered Harmful" Essays Considered Harmful, December 28, 2002.
- M. Nagappan, et. al, "An Empirical Study of Goto in C Code from GitHub Repositories", Proceedings of the 2015 10th Joint Meeting on Foundations of Software Engineering, Pages 404-414
- Using GOTO in Linux Kernel Code, January 13, 2003,
Oct 15, 2016
Today marks the 60th anniversary of the release of the original Fortran Programmer's Reference Manual. Fortran was the world's first high-level computer programming language, was developed beginning in 1953 at IBM by a team lead by John Backus. The first compiler was released in 1957. According to the manual, one of the main features was:
Object programs produced by FORTRAN will be nearly as efficient as those written by good programmers.
The first Fortran compiler (which was also the first optimizing compiler) was named one of the top 10 algorithms of the 20th century:
The creation of Fortran may rank as the single most important event in the history of computer programming: Finally, scientists (and others) could tell the computer what they wanted it to do, without having to descend into the netherworld of machine code. Although modest by modern compiler standards—Fortran I consisted of a mere 23,500 assembly-language instructions—the early compiler was nonetheless capable of surprisingly sophisticated computations. As Backus himself recalls in a recent history of Fortran I, II, and III, published in 1998 in the IEEE Annals of the History of Computing, the compiler "produced code of such efficiency that its output would startle the programmers who studied it."
The entire manual was only 51 pages long. Fortran has evolved significantly since the 1950s, and the descendant of this fairly simple language continues to be used today. The most recent version of the language (a 603 page ISO standard) was published in 2010.
See also
- John W. Backus, New York Times obituary, March 20, 2007.
- J. Backus, "The History of FORTRAN I, II, and III", ACM SIGPLAN Notices - Special issue: History of programming languages conference, Volume 13 Issue 8, August 1978, Pages 165 - 180.