mscharhag, Programming and Stuff;

A blog about programming and software development topics, mostly focused on Java technologies including Java EE, Spring and Grails.

  • Thursday, 23 July, 2020

    JSON Schema validation in Java

    In this post we will see how to validate a JSON document against a JSON Schema in Java. We will use the same JSON document and Schema as in the previous post about JSON Schema.

    You can find both as text files on GitHub: JSON document and JSON Schema.

    We use the networknt JSON Schema validator library in this example. This library seems like a good fit because it supports the latest JSON Schema version (2019-09) and uses Jackson as JSON library. This makes it easy to integrate JSON Schema validation in Spring (hint: upcoming blog post).

    We need to add the following dependency to our project:

    <dependency>
        <groupId>com.networknt</groupId>
        <artifactId>json-schema-validator</artifactId>
        <version>1.0.42</version>
    </dependency>

    Now we can validate our JSON document in Java:

    private static InputStream inputStreamFromClasspath(String path) {
        return Thread.currentThread().getContextClassLoader().getResourceAsStream(path);
    }
    
    public static void main(String[] args) throws Exception {
        ObjectMapper objectMapper = new ObjectMapper();
        JsonSchemaFactory schemaFactory = JsonSchemaFactory.getInstance(SpecVersion.VersionFlag.V201909);
    
        try (
                InputStream jsonStream = inputStreamFromClasspath("example.json");
                InputStream schemaStream = inputStreamFromClasspath("example-schema.json")
        ) {
            JsonNode json = objectMapper.readTree(jsonStream);
            JsonSchema schema = schemaFactory.getSchema(schemaStream);
            Set<ValidationMessage> validationResult = schema.validate(json);
    
            // print validation errors
            if (validationResult.isEmpty()) {
                System.out.println("no validation errors :-)");
            } else {
                validationResult.forEach(vm -> System.out.println(vm.getMessage()));
            }
        }
    }

    When obtaining a JsonSchemaFactory we need to pass a VersionFlag. This defines the JSON Schema version we want to use (here: 2019-09).

    We then use a small helper method to load both files from the classpath. A Jackson ObjectMapper instance is used to read the JSON data from the InputStream and parse it into a JsonNode object. From the JsonSchemaFactory we can obtain a JsonSchema object which can then be used validate the JsonNode. In case of validation errors the returned Set will contain one or more ValidationMessage objects. If the returned Set is empty, no validation errors were found.

    If we accidentally set the painting height to a negative number in our JSON document, we will get the following validation message:

    $.dimension.height: must have a minimum value of 1

    You can find the example source code on GitHub.

  • Tuesday, 21 July, 2020

    Validating and documenting JSON with JSON Schema

    JSON Schema is a way to describe a JSON document. You can think of XML Schema for JSON. It allows you to define required elements, provide validation constraints and documentation.

    I think the easiest way to explain JSON Schema is to look at an example snippet of JSON and the corresponding JSON Schema. So, I created an image that shows both and is hopefully self explanatory:

    json-schema

    (Click to enlarge, if you are on a mobile device)

    A JSON Schema validator can be used to validate a JSON document against a JSON Schema. This can be done online (e.g. https://www.jsonschemavalidator.net/) or using a library of your favourite programming language. In the implementations section of json-schema.org you can find a couple of libraries to work with JSON Schema.

    In case you want to copy/paste some of the sections of the image: You can find the example as text on GitHub.

  • Wednesday, 15 July, 2020

    Common HTTP Status codes

    What is a HTTP status code?

    HTTP status codes are part of the status-line of a HTTP response. These 3-digit integer codes indicate the result of the servers attempt to satisfy the request.

    The first digit of the status-code is used to categorize the response:

    • 1xx: Informal
    • 2xx: Success, the request has been understood and accepted
    • 3xx: Redirection, further action needs to be taken
    • 4xx: Client error, there was a problem with the request
    • 5xx: Server error, the request has been accepted, but the processing failed due to a server error

    Commonly used HTTP status codes

    Here is a list of status codes commonly used in web applications and REST APIs.

    HTTP 200 OK

    The request has succeeded. HTTP 200 is often used as the default status code to indicate that everything worked as expected. However, other 2xx status code can be more specific in certain situations.

    HTTP 201 Created

    The request has succeeded and a new resource has been created. HTTP 201 is often used after a resource has been created by a POST or PUT request (see REST: Creating resources). The Location header field can be used to return the URI(s) of the newly created resource. Note that the resource must be created before 201 is returned. If the resource is created later by an asynchronous process, the server should respond with HTTP 202 (Accepted, see below).

    HTTP 202 Accepted

    The request has been accepted, but the processing has not been completed. For example, the server might have saved the request, but the processing will be done later by a batch job. It is also possible that the request might be disallowed when processing actually takes place. In this case, the request has no effect. The response should include some indication about the processing status. It can also be a good idea to return a reference to a resource where the client can get the current processing status and the result once processing has been finished.

    HTTP 204 No Content

    The request succeeded, but there is no content to send for this request. When sending 204 the response body must be empty. Updated meta-information can be passed in response headers.

    HTTP 304 Not Modified

    This status code is used for caching purposes when a client issued a conditional GET request. Such a request has to contain a If-None-Match or If-Modified-Since header. 304 is returned with an empty response body to indicate that the resource has not been modified. In case the resource has been modified, the resource should be returned with a status code 200 (OK).

    HTTP 307 Temporary Redirect

    The URI of the target resource has been temporary changed and current URI is given in the response Location header. Temporary Redirect indicates that the client should use the original request URI for future requests.

    HTTP 308 Permanent Redirect

    Like 307 this status code is used when the target resource URI has been changed. The new URI is again given in the Location header. However, 308 indicates that the URI change is permanent and clients should use the updated URI for future requests.

    HTTP 400 Bad Request

    The request has been received, but the server is unable to process it due to malformed request syntax. The client should not repeat the request without modification. This status code is often returned if server side validation of the request data fails.

    HTTP 401 Unauthorized

    The request lacks credentials and cannot be authenticated. Note that this status code is badly named, it is used to indicate missing authentication not missing authorization. For missing authorization HTPP 403 is used. This status code is typically returned if the request does not include required authentication information such as passwords or tokens.

    HTTP 403 Forbidden

    The client is authenticated, but does not have the permission to use the given request method on the requested resource. It is also viable to respond with HTTP 404 (Not Found) in these situations if the server wants to hide the resource. The client should not repeat the request (without changing credentials). If the client is not authenticated at all, HTTP 401 should be returned instead.

    HTTP 404 Not Found

    The server has not found the requested URI and is therefore unable to process the request.

    HTTP 405 Method Not Allowed

    The server does not allow the requested HTTP method for the given URI. For example, the request specifies the PUT method for a resource that only supports GET and POST. The response must include an Allow header containing a list of valid request methods for the requested resource.

    HTTP 409 Conflict

    The request could not be completed due to a conflict with the current state of the resource. This code must only be used in situations where the user might be able to resolve the conflict and reissue the request. The response body should contain information about the conflict so the client is able to solve it. An example can be a resource update using PUT. Maybe the resource has been updated by another third-party request and the current request does not reflect the current resource state.

    HTTP 500 Internal Server Error

    The server has encountered a situation it is unable to handle. This is the standard status code for unexpected error on the server during the request processing.

     

  • Monday, 13 July, 2020

    Quick tip: ISO 8601 durations in Java

    Many developers know about the interchange formats for dates and times defined by ISO 8601. (For example 2007-08-31T16:47+00:00 which represents 16:47 on August 31, 2007 in UTC)

    However, what is not so well-known (at least in my experience), is that this standard also defines a format for durations.

    Here are a few examples:

    • P1Y - 1 year
    • P2M4D - 2 months and 4 days
    • P3Y6M4DT12H30M5S - 3 years, 7 months, 4 days, 12 hours, 30 minutes, and 5 seconds

    In Java it is quite easy to work with this format because the java.time API can automatically parse it to Duration and Period objects.

    Duration is used to work with hours and smaller time units while Period works with dates.

    For example:

    Duration duration = Duration.parse("PT12H"); // 12 hours
    Period period = Period.parse("P3Y6M"); // 3 years and 6 months

    We can now work with these instances, for example we can add period to a LocalDate:

    LocalDate result = LocalDate.of(2020, 8, 1).plus(period); // 2024-02-01
  • Wednesday, 8 July, 2020

    Getting started with Ktor

    Ktor (pronounced kay-tor) is an open source Kotlin framework for building asynchronous web applications. This post shows how to create a small RESTful CRUD service with Ktor.

    Getting started

    In this example we use Maven as build tool. Besides standard Kotlin dependencies we need to add the Ktor dependencies to our pom.xml:

    <project>
    
        <properties>
    		...
            <ktor.version>1.3.2</ktor.version>
        </properties>
    
        <repositories>
            <repository>
                <id>jcenter</id>
                <url>https://jcenter.bintray.com</url>
            </repository>
        </repositories>
    
        <dependencies>
            <dependency>
                <groupId>io.ktor</groupId>
                <artifactId>ktor-server-core</artifactId>
                <version>${ktor.version}</version>
            </dependency>
            <dependency>
                <groupId>io.ktor</groupId>
                <artifactId>ktor-server-netty</artifactId>
                <version>${ktor.version}</version>
            </dependency>
            <dependency>
                <groupId>io.ktor</groupId>
                <artifactId>ktor-gson</artifactId>
                <version>${ktor.version}</version>
            </dependency>
            <dependency>
                <groupId>ch.qos.logback</groupId>
                <artifactId>logback-classic</artifactId>
                <version>1.2.3</version>
            </dependency>
        </dependencies>
    	...
    </project>

    You can find the full pom.xml (including standard Kotlin dependencies and plugins) on GitHub.

    For building a server application we need ktor-server-core. We use Netty as web server and therefore add ktor-server-netty. To work with JSON requests/responses we use Ktor's GSON integration with ktor-gson. Ktor uses SLF4J for logging and needs a SLF4J provider, we use logback-classic.

    To see if Ktor is ready, we can run the following piece of code:

    fun main() {
        embeddedServer(Netty, 8080) {
            routing {
                get("/foo") {
                    call.respondText("Hey", ContentType.Text.Html)
                }
            }
        }.start(wait = true)
    }

    This defines a single endpoint that listens for GET requests on /foo. Running the main method starts an embedded Netty server listening on port 8080. We can now open http://localhost:8080/foo in a web browser and should see the text Hey as response.

    Configuring JSON handling

    Next we need to set up JSON handling in Ktor. We do this by adding the ContentNegotiation feature with GSON converters to our Ktor application:

    fun main() {
        embeddedServer(Netty, 8080) {
            install(ContentNegotiation) {
                gson {
                    setPrettyPrinting()
                }
            }
    
            routing { 
    			...
    		}
        }.start(wait = true)
    }

    With the install() method we can add a feature to our application. Features can provide additional functionality which can be added to the request and response processing pipeline. The ContentNegotiation feature provides automatic content conversion based on Accept and Content-Type headers. The conversion is handled by GSON.

    This way we can return a Kotlin object from an endpoint and the content conversion will be handled by Ktor. If a client accepts JSON (Accept header with value application/json) Ktor will use GSON to convert the Kotlin object to JSON before it is sent to the client.

    Implementing get operations

    In this example we create a simple CRUD API for products. For this we use two small helper classes: Product and Storage.

    Product is a basic product representation containing a name, a description and an id:

    class Product(
            val id: String,
    		val name: String,
            val description: String
    )

    Storage acts as our simplified product database, it uses a MutableMap to save products:

    class Storage {
        private val products = mutableMapOf<String, Product>()
    
        fun save(product: Product) {
            products[product.id] = product
        }
    
        fun getAll() = products.values.toList()
        fun get(id: String) = products[id]
        fun delete(id: String) = products.remove(id)
    }

    With these two classes we can now start to create our Ktor endpoints. Of course we can just add more routes to the main method we used above. However, we typically want to separate different types of routes into different files. Therefore, we create our own Route extension function, named product() in a separate file:

    fun Route.product(storage: Storage) {
        route("/products") {
    
            // GET /products
            get {
                call.respond(storage.getAll())
            }
    
            // GET /products/{id}
            get("/{id}") {
                val id = call.parameters["id"]!!
                val product = storage.get(id)
                if (product != null) {
                    call.respond(product)
                } else {
                    call.respond(HttpStatusCode.NotFound)
                }
            }
    	}
    }

    This defines two GET operations below the /products route. One to obtain all products and one to obtain a single product by id. In the single product endpoint we use call.parameters["id"] to obtain the product id from the request URI. With call.respond() we can define the response we want to send to the client.

    Next we have to create our product storage and add our routes to the server. We do this by calling our Route extension function within the routing section of our server:

    fun main() {
        embeddedServer(Netty, 8080) {
            install(ContentNegotiation) {
                gson {
                    setPrettyPrinting()
                }
            }
            routing {
                val productStorage = Storage()
    
                // add some test data
                productStorage.save(Product("123", "My product", "A nice description"))
                productStorage.save(Product("456", "My other product", "A better description"))
    
    			// add product routes
                product(productStorage)
            }
        }.start(wait = true)
    }

    We can now run this application and send a GET request to http://localhost:8080/products. We should get a JSON array containing two products as response:

    Request:

    GET http://localhost:8080/products

    Response:

    [
      {
        "id": "123",
        "name": "My product",
        "description": "A nice description"
      },
      {
        "id": "456",
        "name": "My other product",
        "description": "A better description"
      }
    ]

    Implementing create and update operations

    To create a new product, we use the POST operation on /products. Once a product has been created, we respond with HTTP 201 (Created) and a Location header which points to the newly created product. Our Ktor code looks like this:

    fun Route.product(storage: Storage) {
        route("/products") {
    		...
    	
    		// POST /products
    		post {
    			val data = call.receive<Product>()
    			val product = Product(
    					id = UUID.randomUUID().toString(),
    					name = data.name,
    					description = data.description
    			)
    			storage.save(product)
    			call.response.headers.append("Location", "http://localhost:8080/products/${product.id}")
    			call.respond(HttpStatusCode.Created)
    		}
    	}
    }

    With call.receive() we can convert the request body JSON to a Product object. Note that we use the data object from call.receive() to create another product object that is then passed to storage.save(). If we would pass data directly to storage.save() it would be possible for the client to define the product id. Like the other fields, id will be mapped from JSON into the Product object with call.receive(). However, we want to make sure we get a new server generated product id. An alternative solution would be to create a specific transfer object that does not contain the id field.

    With call.response.headers.append() we add Location header to the response.

    The following snippets show an example for a possible request/response combination:

    Request:

    POST http://localhost:8080/products
    Content-Type: application/json
    {
        "name": "New product",
        "description": "This product is new"
    }

    Response:

    201 (Created)
    Location: http://localhost:8080/products/70f01c0f-0a6f-4227-a064-02383590a5ab

    A product is updated by sending a PUT request to /products/{id}. The Ktor endpoint looks like this:

    fun Route.product(storage: Storage) {
        route("/products") {
    		...
    
            // PUT /products/{id}
            put("/{id}") {
    			val id = call.parameters["id"]!!
                val product = storage.get(id)
                if (product != null) {
    				val data = call.receive<Product>()
                    product.name = data.name
                    product.description = data.description
                    storage.save(product)
                } else {
                    call.respond(HttpStatusCode.NotFound)
                }
            }
    	}
    }

    Again we use call.parameters and call.receive() to get the product id and the content from the request body. We check if a product with the given id is available. If this is the case we update the product otherwise HTTP 404 (Not Found) is send to the client.

    Deleting products

    We delete products by sending a DELETE request to /products/{id}. The endpoint implementation does not use any new Ktor features, but for sake of completeness here is it:

    fun Route.product(storage: Storage) {
        route("/products") {
    		...
    
            // DELETE /products/{id}
            delete("/{id}") {
                val id = call.parameters["id"]!!
                val product = storage.delete(id)
                if (product != null) {
                    call.respond(HttpStatusCode.OK)
                } else {
                    call.respond(HttpStatusCode.NotFound)
                }
            }
        }
    }

    Conclusion

    Ktor provides an easy way to build a REST API with Kotlin. It uses Kotlin Lambdas to define routes and endpoints. We learned how to access request information, send JSON responses, set response headers and a few other things by building this small CRUD example service. You can find the example code on GitHub.