These forums have been archived and are now read-only.

The new forums are live and can be found at https://forums.eveonline.com/

EVE Technology Lab

 
  • Topic is locked indefinitely.
 

CCP employees and #devfleet - should json be object oriented?

Author
Salgare
Center for Advanced Studies
Gallente Federation
#1 - 2016-08-26 02:41:22 UTC  |  Edited by: Salgare
Hey all, there was a lively discussion on #devfleet today about object oriented json structure (and CCP's lack their of). Things flew by so quickly I could not keep up with the dogs piling on. Thus I'm hoping to capture the thinking of various individuals here where we can stop and think about each others points of view.

Too often I get stuck in my own pattern of thinking and I'm interested to learn if the way my current paying job designs and implements json objects is not optimal for other languages that I'm not familiar with. I would like to understand the non-typed and auto-typed languages best practices and see if they conflict with strongly typed language best practices.

The focus of this discussion is to try and determine what the industry considers good json practices. Since json is platform and language independent, this discussion should not drift into the pro's and con's of languages or platforms.

I'd like to use the crest servers base url list of crest endpoints as an example.

https://crest-tq.eveonline.com
https://crest-tq.eveonline.com/schema

We provide a shadow schema tree similar to the second link example above. I don't necessarily want to deep end on schema trees and just mention it here as a comparison to the following sudo code example of what I would declare for the crest endpoints root url call.

sudo json schema for crest endpoints list
what I would do follows:

 
"{}" = delimits an object
"[]" = delimits a variable sized list
":" = delimits label from value
"," = delimits objects in a list as well as child objects
"()" delimits a type

{
    "endpoints":{
         "serverName"(String):, "serverVersion"(String):, "userCount"(Integer), "serviceStatus"(Boolean):[{
             (EndpointGroup)
                 "groupName"(String):[
                     (Endpoint){
                         "name":(String),"uri":(String)
                     }
                 ]
             }
         ]
    }
}


If ccp had a schema tree, this is what this data object graph would be declared as:
correction, from Admiral Blue in following post. crest has "option" data available that can be used to generate proper schema

{
    "serverVersion"(String):,
    "userCount_str"(String):,
    "userCount"(Integer):,
    "serverName"(String:),
    "serviceStatus"(String): ,

    "constellations":{"href"(String):},
    "itemGroups":{"href"(String):},
    "corporations":{"href"(String):},
    "alliances":{"href"(String):},
    "decode":{"href"(String):},
    "marketPrices":{"href"(String):},
    "opportunities":{"href"(String):,
        "tasks":{
            "href"(String):
        },
        "groups:{
            "href"(String):
        }
    },
    "itemCategories":{"href"(String):},
    "regions":{"href"(String):},
    "bloodlines":{"href"(String):},
    "marketGroups":{"href"(String):},
    "systems":{"href"(String):},
    "sovereignty":{"href"(String):, "campaigns":{"href"(String):}, "structures":{"href"(String):}},
    "tournaments":{"href"(String):},
    "virtualGoodStore":{"href"(String):},
    "wars":{"href"(String):},
    "incursions":{"href"(String):},
    "dogma":{"href"(String):, "attributes":{"href"(String):}, "effects":{"href"(String):}},
    "races":{"href"(String):},
    "insurancePrices":{"href"(String):},
    "authEndpoint":{"href"(String):},
    "itemGroups":{"href"(String):},
    "industry":{"href"(String):, "facilities":{"href"(String):}, "systems":{"href"(String):}},
    "npcCorporations":{"href"(String):},
    "time":{"href"(String):},
}



sudo application data object

For my schema

Application object
    var endpoints type is of Endpoints
Endpoints object
    var serverName type is of String
    var  List of type is of EndpointGroup
EndpointGroup object
     var groupName type is of String
     var List of type is of Endpoint
Endpoint object
    var name type is of String
    var uri type is of String 

// to access
endpoints.List[i].groupName.List[j].uri


sudo application data object
For ccp schema

Application object
    var map type is of HashMap
HashMap object
   var key type is of String
   var key type is of no-type, or auto-type

//to access
if HashMap of HashMaps
map["wellKnownLabel-level3"].map["wellknownLabel-level2"].map["wellknownLabel-level1"] ...
if single HashMap
key = WellKnownLabel-level1 + "." + WellKnownLabel-level3 + "." WellKnownLabel-Level3
map[key ]



Example snippet of what my json schema would look like


{
    "serverName": "TRANQUILITY",
    "serverVersion": "EVE-TRANQUILITY 14.07.1066627.1066627",
    "userCount_str": "25155",
    "serviceStatus": "online",
    [
        ...,
        {"groupName": "industry",
            [
                {"name": "facilities", "uri": "https://crest-tq.eveonline.com/industry/facilities/"},
                {"name": "systems", "uri": "https://crest-tq.eveonline.com/industry/systems/"}
            ]
        },
        {"groupName": "npcCorporations",
            [{"name": "npccorps", "uri": "https://crest-tq.eveonline.com/corporations/npccorps/"}]
        },
        ...
    ]
}


Please discuss
Salgare
Center for Advanced Studies
Gallente Federation
#2 - 2016-08-26 02:57:06 UTC  |  Edited by: Salgare
Admiral Blue hooked me up with the following video from one of the original authors.

http://jimpurbrick.com/2016/01/03/crestmatic/

The takeaway for me from this link:

"Fortunately CREST is self describing: send an OPTIONS request to a CREST URI and a list of methods and representations that can be used with that URI is returned. The metadata isn’t in a standard format, so I built the crestschema library which can convert the CREST format in to more useful JSON schemas."
Kolimara
New Eden Trading Association
#3 - 2016-08-26 03:58:54 UTC
Well.. i wrote a library-generator for CREST some time ago.

Currently it only produces Haskell-Code (with parsers etc.), but you can generate anything you like from that.

In its current version it only takes the root-path and parses the OPTION-Request - but it should be trivial to filter the list for URLs and recurse with OPTION-Requests down the rabbit-hole (while keeping book on what we have visited so far to not loop endlessly..).

URL: https://github.com/Drezil/crest-apigen
Salgare
Center for Advanced Studies
Gallente Federation
#4 - 2016-08-26 04:54:48 UTC
Kolimara wrote:
Well.. i wrote a library-generator for CREST some time ago.

Currently it only produces Haskell-Code (with parsers etc.), but you can generate anything you like from that.

In its current version it only takes the root-path and parses the OPTION-Request - but it should be trivial to filter the list for URLs and recurse with OPTION-Requests down the rabbit-hole (while keeping book on what we have visited so far to not loop endlessly..).

URL: https://github.com/Drezil/crest-apigen


I have not driven down past the root level yet, and just learned about this OPTION request.

cool beans on writing a code generator.

What we sometimes shoot for is a generic piece of code that dynamically figures out the interface at run time. In Java that concept is often referred to as a BeanBox, where the box, given a well defined Bean establishes the json to Java Object mapping dynamically at runtime. Write once, run anywhere. The Bean in this case is the proper json schema. I assume the returned OPTION data is identical for any/all nodes and is a small amount of data needing mapped.
Messenger Of Truth
Butlerian Crusade
#5 - 2016-08-26 11:21:48 UTC  |  Edited by: Messenger Of Truth
Quote:

Hey all, there was a lively discussion on #devfleet today about object oriented json structure (and CCP's lack their of).


What exactly do you mean by object-oriented here? If you're asking for a more precise definition of the CREST endpoints, that's not got a lot to do with object-orientation. So are you using the words correctly?

Trade Hub Price Checker: stop.hammerti.me.uk/pricecheck

Visit "Haulers Channel" in game for all matters courier-related.

Structure name/system API: stop.hammerti.me.uk/api

Diana Olympos
Deep Core Mining Inc.
Caldari State
#6 - 2016-08-26 12:02:48 UTC
OK i have a couple things to say here. Firstly, i am not sure i really grok your question, so i will try to answer the different scenarios i can imagine.

So before going deeper, i would advise you to have a look at this explanation about types, it will probably make a lot of things cleaner. In particuler the stuff about stong and weak typing A quick summary of types

Now to dive into JSON. Let see. The most common usage of JSON is as a data transport. It is quite obvious and it is its goal after all : be a way to transfer data between point A and point B, with a known format.

So here is how i would decode JSON. I am relying on the fact you have Union Types, a HashMap structure with Accessors and a Array structure with Accessors. I am also relying on having a String type that can deal with any binary (ie not C Strings).
Then Json structure is quite simple

defstruct Node_JSON
  key Type(String) : Value Type(Hashmap of Node_JSON | Array of Node_JSON | String | Boolean | Number | null)


defstruct JSON
  Hashmap Type(NodeJSON)



Now all we have to do is to feed that to a parser and done. You can access any element with
your_parsed_JSON["key"]


No need for something more complex. Now that is sound and all, but i have the feeling it is not really the use case you are asking about.

What if you want to wrap CREST behind an API, automatically walking it and preloading stuff, to provide a nice piece of data while hiding the fact it is coming from Crest. Well then you enter a different place. You can do it at runtime like you said earlier, but well... we all know how nice it is to use things defined at runtime through the internet. Nothing can go wrong right...

Or you can do it at compile time. But it means you will be stucked with the version (yes CREST is versionned... well... when they use it...). Which is not a completely bad thng, because it mean you will also only keep the data you know you can handle and reject the rest. But once again, no need for Object for that. You can just define a data struct.

So in the end : sure if you want to map CREST data to some object you create to recreate a data source by hiding CREST and providing your own methods, aggregation and different accesor, then use JSON Schemas and deal with all that. You can also use JSON Schema to define what you expect and check it.

But in the end, my real question is : why do you need objects like that?


PS : You may not Union Types usually, but they can be implemented in any languages. I have at least one idea of how to implement them naively (and poorly) so it
Salgare
Center for Advanced Studies
Gallente Federation
#7 - 2016-08-26 15:15:27 UTC  |  Edited by: Salgare
Messenger Of Truth wrote:

What exactly do you mean by object-oriented here?


Yes, I should have defined terms first. In this case (json) I'm referring to simply modeling of data (no behaviors) to there real world structures. Years back we used to say try and describe it in english with "isa" "hasa" and "uses". In pure data objects "uses" does not pertain. Consider this:

A Vehicle
A Car isa Vehicle
A Engine
A FourCylinder isa Engine
A Car hasa engine
A Ford isa Car
A Fiesta isa Ford
A Fiesta hasa FourCylinder
etc.


don't they teach you boys and girls UML anymore? Cool

In the op I only modeled my example after a simple "node" object that has a "list" of its children. To keep things a bit simpler I did not start as I typically would with the "composite pattern" and declare a tree, which in the case of the op example is the root of the tree, the thing we are "modeling to objects". Thus:
In the op a full suggested schema might look like this:

Crest hasa GroupedEndpoints - 1:many
    GroupedEndpoints
        hasa name
        hasa Endpoint -1:many
    Endpoint
        hasa name
        hasa uri
        hasa Endpoint 1:many


Quote:

If you're asking for a more precise definition of the CREST endpoints, that's not got a lot to do with object-orientation. So are you using the words correctly?


Hopefully the above example shows that indeed crest endpoints describe real life things (a tree in above example) that indeed can (and should) be modeled in json in an object model.

Note two other OO terms I'm likely to use:

"Class Graph" = the class that declares the data its types and structure ... i.e. the english example above or a Java class.
"Object Graph" = A given instantiate of a Class Graph .. i.e. the actual data, the whole tree in this op/example.

There is an offical json schema to declare Class Graphs (typically provided in a shadow/parallel tree to data root ).
Salgare
Center for Advanced Studies
Gallente Federation
#8 - 2016-08-26 15:57:17 UTC
Quote:
Now to dive into JSON. Let see. The most common usage of JSON is as a data transport. It is quite obvious and it is its goal after all : be a way to transfer data between point A and point B, with a known format.


More direct question: Would the "my way" schema/examples of the OP be non-optimal for functional languages, php, python etc. If so, how and why?

Actually another goal of json is to provide a human friendly readable form as well. I would suggest current ccp endpoints don't meet this goal as they are not modeled as data objects after the real world (which are natural to humans).

Quote:

So here is how i would decode JSON. I am relying on the fact you have Union Types, a HashMap structure with Accessors and a Array structure with Accessors. I am also relying on having a String type that can deal with any binary (ie not C Strings).
Then Json structure is quite simple

defstruct Node_JSON
  key Type(String) : Value Type(Hashmap of Node_JSON | Array of Node_JSON | String | Boolean | Number | null)
defstruct JSON
  Hashmap Type(NodeJSON)



I have bolded two parts of your above statement. Both of these violate the goal of human friendly readable. Realize that often times it is NOT a programmer that has to deal with the raw json (IT guys configuring things for example).
As for Unions they are typically a programming level optimization that obfuscates usage were a properly declared object model clearly declares the usage.

The "quite simple", where does this requirement come from? Who is it simple for? If I'm really modeling data for generic use by unknown consumers should I model/meta-data the Class Graph thoroughly (professionally) or are we doing a simple quick and dirty implementation friendly to a given group of programing languages?

Quote:

Now all we have to do is to feed that to a parser and done. You can access any element with [code]your_parsed_JSON["key"]
No need for something more complex.

Remember we are not discussing languages, platforms, given protocols (note I did not use the label "href") etc.
And there is a reason for more complexity if you are doing a professional modeling of your data.
[
Quote:

Now that is sound and all, but i have the feeling it is not really the use case you are asking about.

Lets not deep end on the uses of a shadow schema tree (i.e. proprietary options request in ccp). I'm not asking how to's only discussion why I complain about ccp's json model and are those complaints valid to other languages than java.

ccp already gets a strike one for not providing json standard schema

Salgare
Center for Advanced Studies
Gallente Federation
#9 - 2016-08-26 17:19:14 UTC  |  Edited by: Salgare
Would someone please show me a python or php code snippet that would load this json snippet and then access root.industry.systems and explain/comment the python/php snippet for a python/php dummy like myself?


{
    "serverName": "TRANQUILITY",
    "serverVersion": "EVE-TRANQUILITY 14.07.1066627.1066627",
    "userCount_str": "25155",
    "serviceStatus": "online",
    [
        ...,
        {"groupName": "industry",
            [
                {"name": "facilities", "uri": "https://crest-tq.eveonline.com/industry/facilities/"},
                {"name": "systems", "uri": "https://crest-tq.eveonline.com/industry/systems/"}
            ]
        },
        {"groupName": "npcCorporations",
            [{"name": "npccorps", "uri": "https://crest-tq.eveonline.com/corporations/npccorps/"}]
        },
        ...
    ]
}


Also, a comparison python/php snippet for the corresponding ccp json snippet would be helpful.

{
"industry":
{
    "facilities":
    {
        "href": "https://crest-tq.eveonline.com/industry/facilities/"
    },
    "systems":
    {
        "href": "https://crest-tq.eveonline.com/industry/systems/"
    }
},
}


java might look like this

        Gson gson = new Gson();  // get the google json parser
        CrestCallTree root = gson.fromJson(jsonBody, CrestCallTree.clazz);  // use gson to parse jsonBody into CrestCallTree object
        EndpointGroup endpointGroup = root.endpointGroups.get("industry");  // get the industry call group
        Endpoint endpoint = endpointGroup.get("systems");  // we now have root.industry.endpoint
        logger.info(endpointGroup.groupName + "." + endpoint.name + ".url = " + endpoint.url);

// result , "industry.systems = https://crest-tq.eveonline.com/industry/systems/" is logged.
Salgare
Center for Advanced Studies
Gallente Federation
#10 - 2016-08-26 20:55:40 UTC  |  Edited by: Salgare
I need to acknowledge something at this point. Admiral Blue pointed out to me on #devfleet that CCP was a very early adopter of REST. I took special notice of this at that time as our group was getting heavy into at that time also. Things were not well supported and fully defined at that time. It's the penalty one pays for going with bleeding edge technologies.

So, in hind site, I give big kudo's to CCP for taking the leap and apologize for speaking so negatively about it. It's rumored that the future is bright in this arena. Part of my goal was to encourage that, but it sounds like I may have been singing to the CCP choir.

So, that only leaves my one other purpose of this thread. I have had several discussions with Diana about OO, OOA, OOP of which we have 180 degree opinions on. I was hoping for a bit of a formal debate here on these issues. I was somewhat shocked when most on #devfleet were declaring that json was not/could not be OO. Thus the op and setup of a simple example of a "real thing" (crest root) being mapped to an Class/ObjectGraph.

So OP change of rules ... lets talk languages and why OO is laughed at by Functional Language Programmers and how/why they say json is not OO ... perhaps detailed descriptions here have changed their minds?

I would be very surprised if FP's did not at least have a story at OOA levels
Salgare
Center for Advanced Studies
Gallente Federation
#11 - 2016-08-26 22:04:33 UTC  |  Edited by: Salgare
Diana,

You expressed confusion on my statement about unions. Perhaps you need to define your usage of the term. My statement was based on the C programming language definition of the term.

It has been many moons since I coded C (10 years of it at Novell) but as I remember it, the concept was one of overlaying two (or more) data over a single pointer with the size of the data pointed to being the minimal required size of the union.

In c, it was used as a form of data inheritance.

Of course C is a low level implementation of the concept, but given your examples I assume we are talking the same thing: i.e. either an array or a class is being overlaid on a single pointer (the variable getting the assignment)

Are we talking the same thing?
Vincent Claeson
The Scope
Gallente Federation
#12 - 2016-08-27 09:47:01 UTC
Salgare wrote:
You expressed confusion on my statement about unions. Perhaps you need to define your usage of the term. My statement was based on the C programming language definition of the term.


I'm not Diana but I suppose that whatever statement was made related to union types[1] (i.e. sum types) from a type theory perspective and not a specific language implementation of them.

In general I don't understand the OP and what "object-oriented" means here. It sounds like you are just asking for a "cleaner" schema and type annotations.

Quote:
So OP change of rules ... lets talk languages and why OO is laughed at by Functional Language Programmers and how/why they say json is not OO ... perhaps detailed descriptions here have changed their minds?


I think there's different things people laugh at here. One is languages like Java that make many trivial tasks needlessly complicated and are difficult/annoying to write without a good IDE due to how verbose they are. When working on Java projects I often do an initial implementation of a service in Erlang/Elixir because I know it will be significantly faster to do that, and then rewrite it slowly in Java when I have time for it.

The other thing is the OOP "way of thinking". In many functional languages we have no classes, inheritance, interfaces, methods attached to objects, objects and so on - we only have data types and functions that operate them.

Instead of defining a tree of objects to be able to provide functionality at some higher level we may use type classes (in Haskell) (also don't get confused by the word "class", it has very little relation to OOP classes) to implement functions that operate on several types.

People who "switch" from OOP languages to functional languages often quickly realise that their old languages were forcing them to do all sorts of stuff that isn't necessary to write the program.

1: https://en.wikipedia.org/wiki/Union_type
Salgare
Center for Advanced Studies
Gallente Federation
#13 - 2016-08-27 11:13:48 UTC  |  Edited by: Salgare
Vincent Claeson wrote:

I'm not Diana but I suppose that whatever statement was made related to union types[1] (i.e. sum types) from a type theory perspective and not a specific language implementation of them.

When I hear "union" my mind goes straight to data sets. Union. It seems strange that FP would use the word to describe either/or typing, but now I know. Thanks for the footnote on "Union Type" , I retract my statement about complexity etc. as I was thinking oranges and Diana was thinking apples. BTW, as java evolved (from c syntax) unions were considered bad and given no place in java, with oo classes being the preferred method.

Quote:

In general I don't understand the OP and what "object-oriented" means here. It sounds like you are just asking for a "cleaner" schema and type annotations.


definition of terms given upthread but not in OP
https://forums.eveonline.com/default.aspx?g=posts&m=6609842#post6609842

No not just cleaner, OO per that definition. Everyone cries about UML (as well as java), but I have to ask, has no one designed complex relational database schema's and not used diagram Modeling tools to design that schema? Have you not had to map out relationships between data as in hasa, isa and multiplicity? 1:1, 1:*, *:* (* == "many")? If one is always only thinking name/value pairs, these are wonderful in some situations, and yet are bad choices in most data models (of any complexity). They are flat structures requiring unique and Well-Known names.

In the OP json example one of ccp's weaknesses lies in this well-known name weakness. To add a new call group a new unique well-known name is introduced, where the suggested OO example has a fixed set of constant name/id's that represent 1:* call groups. i.e. the name of the group is never a key, always a value.

Quote:

I think there's different things people laugh at here. One is languages like Java that make many trivial tasks needlessly complicated and are difficult/annoying to write without a good IDE due to how verbose they are. When working on Java projects I often do an initial implementation of a service in Erlang/Elixir because I know it will be significantly faster to do that, and then rewrite it slowly in Java when I have time for it.


For me to try and understand how FP bypasses the need/desire of OO (for model) and OOA for complex systems, java is irrelevant. I'm hoping to talk theory here more than any given implementation.

Quote:

The other thing is the OOP "way of thinking". In many functional languages we have no classes, inheritance, interfaces, methods attached to objects, objects and so on - we only have data types and functions that operate them.

Instead of defining a tree of objects to be able to provide functionality at some higher level we may use type classes (in Haskell) (also don't get confused by the word "class", it has very little relation to OOP classes) to implement functions that operate on several types.

People who "switch" from OOP languages to functional languages often quickly realise that their old languages were forcing them to do all sorts of stuff that isn't necessary to write the program.

This is the part I'm missing. I cannot see how complex systems with complex subsystems don't end up back to the days of spaghetti code. I'm also missing how you model complex data structures.

Also, in my world , I will need to switch back and forth and write code that inter-operates between languages, with OO json being one example of needed interoperability.
Salgare
Center for Advanced Studies
Gallente Federation
#14 - 2016-08-27 11:15:56 UTC  |  Edited by: Salgare
**** hit quote instead of edit
Diana Olympos
Deep Core Mining Inc.
Caldari State
#15 - 2016-08-27 15:39:25 UTC
Ok let's go...

First of all, let's begin by explaining why Vincent or me are separating Java (and the inhritance, class modelising stuff) from OOP.

Object Oriented Programming is an old thing, that go back to XEROX PARC. It was implemented by SmallTalk and developped mainly by Alan Kay. It has nearly nothing to do with inheritance and classes. Even worse, inheritance and classes were added late in the end of the 80s and where consider an antipattern really quickly.

But SmallTalk was an experimental language, so implementing them and playing with them made sense in SmallTalk.

I highly advise to have a look at The Early History of SmallTalk by Doctor Kay himself.

Here is a quote from the introduction that i find quite interesting :
Quote:
Smalltalk’s design–and existence–is due to the insight that everything we can describe can be represented by the recursive composition of a single kind of behavioral building block that hides its combination of state and process inside itself and can be dealt with only through the exchange of messages.


And no Java is not implementing it. Ideas like getters, setters, inheritance or even RPC are things that should not exist in OOP

Don't get me wrong, the idea behind OO (which is to keep state local) is really powerful for everything that have to deal with the outside world of an app, especially things like GUI. But you do not need classes and inheritance for that, nor complex models.

So now that this is done... let's dive in the next thing. How to model a complex thing. Well. First question. Do you really need it ? Once you get rid of the fact you have to deal with state and inheritance, what is left ?

  1. Data. That come in and out.
  2. The structure of that data. (hellooooo Type theory)
  3. The transfomation you apply on that data. (here you would talk about algorithms and function and eventually patterns, but these things are far less numerous that you can think about)
  4. And at most, a way to do polymorphism, so that you can apply a function and a way to think to different data structures so you are not completely locked to a single specific data structure.
  5. A way to link these functions together and to send data to your inputs and outputs


Well... and that's it. So let see how we could do that...

A quite good answer to that is to borrow directly from the UNIX philosophy (which seems to work quite well for OS :D ).

Do things that do one thing.
Combine them.
Use a good way to make them talk to each other.

And that is all. The best way to model a complex problem is to break it into simple components with well defined interface and contracts. Once it is done, you can then use them as bricks to build the level on top of it. And so on and so forth.

A good and simple example of that would be the good old SmallTalk inspired HyperCard.

I am not that good at talking about it, so i will link John Hughes, who have far more experience and talk far better about it.

BTW here is the paper he talk about with the fun little part where a student rewrote the program after only a week of Haskell and was still the fastest one

Here is a greater explanation that in can on Haskell vs OOP

In the end, my question stay the same. Why do you need all that?

Try to think how you could do it without all the stuff of OOP. How you would do it with just functions and data.

Finally : i can find a ton of work by researcher and all on making progress to CS and Software Engineering by the people from the Actor Model, the Imperative world or the Lambda Calculus world. I don't see a lot of things about how Classes and Inheritance make seems faster/easier to work with/better.

PS : about Relational DB model : Sure you model that, because it is about Relation. But we are talking of data there, no more about modeling a program. Plus something that has proven hard time and time is to link Objects into Relational model. Which explains why ORM are always complex and rarely great.
Salgare
Center for Advanced Studies
Gallente Federation
#16 - 2016-08-27 19:24:47 UTC  |  Edited by: Salgare
Diana and Vincent;

First of all, thank you both for taking the time to respond. Slack #devfleet has introduced me to a whole new (actually way old) paradigm and I appreciate this resource very much.

What is crazy to me is I've been programming professionally, no breaks in employment since around 1990. During this time, whisperings of lisp and smalltalk have always arose in theoretic discussions within the industries I've worked in. However there has never been practical applications appearing in those industries where there was any job market for them. The industries of which I speak are (from earliest to latest) - Computer Graphics (Evans and Sutherland 7 years), Industrial Automation (self employed 5 years), File Server OS (Novell 10 years) and Embedded devices to Cloud systems (Panasonic 8 years)

I watched the John Hughes presentation with great interest. It appears FP has found practical application in pretty printers, testing algorithms and chip manufacturing. The presentation highlights how different and difficult FP theory is for me. There has not been a paying job in my 30 years that has had any demand for FP. I believe the explosion of Big Data in the last few years is changing that for one subsystem of my complex OOA world, namely data reduction and the power of a pattern matching language. Scala appears to be the best technology choice for my industry needs in this regard.

Diana, I believe I do have some grasp of how FP replaces (if that's the right word) OOP concepts of inheritance, encapsulation etc. from the little bit I played with Elixir. However examining the links you provided still has not addressed how the complexity of large systems/sub-systems is controlled with FP.

For example from the given haskell link:

Quote:

The great benefit of Object Oriented Programming is rarely that you group your data with the functions that act upon it together into an object - it's that it allows for great data encapsulation (separating the interface from implementation) and polymorphism (letting a whole set of data types behave the same way). However:
Data encapsulation and polymorphism are not exclusive to OOP!


This statement implies the only advantage of OOP is in its data encapsulation and polymorphism, but it does not reflect how these OO classes provide maintainability isolation and extensibility based on loose coupling with other OO classes (avoid spaghetti code). It also does not reflect the power of Dynamically Modules (see https://www.osgi.org/). The OP used a very simple json example of remote interoperability of data, where Dynamic Modules are complete code libraries. see https://www.amazon.com/Design-Patterns-Elements-Reusable-Object-Oriented/dp/0201633612 for MVC and 22 other patterns

Thus I still question how does FP handle the issue of complexity? Perhaps understanding how they deal with code reuse might help. Or how one starts out in a design phase to layout the needed, say Model, View and Controller patterned subsystems, with all of those subsystems being extensible.

Quote:

PS : about Relational DB model : Sure you model that, because it is about Relation. But we are talking of data there, no more about modeling a program. Plus something that has proven hard time and time is to link Objects into Relational model. Which explains why ORM are always complex and rarely great.


In my whole career I have never used an ORM system. Complexity, Performance, Scalability and Extensibility requirements have precluded their use. I assume this might be a bit of a shortcoming for those who have always allowed other libraries to totally handle their persistence for them, thus a possible reason for short-sightedness in say json best practices in some implementations.
Diana Olympos
Deep Core Mining Inc.
Caldari State
#17 - 2016-08-27 20:19:00 UTC
So couple things.


  1. FP is used in a ton of different industry from chip manufacturing to web stuff. There is a post somewhere explaining that their secret weapon was LISP. During the dotcom bubble... by Paul Graham. It could also be argued that JS is a functional language, but this one is a bit of a weak exemple. Rieman is a great alerting service built on a LISP. And i can probably extract other Clojure examples like this one

  2. Most of this complexity do not have to exist !! It only exists because you are using a poor type system (classes) that makes all you are doing a complex thing.

  3. Data encapsulation can be achieve by a ton of way. A closure as an example.

  4. Polymorphism used to be OO forte. But sadly, OO lost that edge when new ways to handle polymorphism evolved from type theory. Dataclasses (also knows as Traits or Protocols) are an exemple of that polymorphism. And they end up be far easier to handle.

  5. So Java worked for you. That make sense. Mainly because computer power exploded, so all the power you were wasting in doing things in an "inefficient" way was hidden by more machine. And it was better than COBOL :p But now that there is no more free lunch all the problems are coming back. The need to have 10 time the number of develloper you should need, the problem of maintainability, the brittleness of inheritance, etc are biting the back of everyone.

  6. About maintenance and reusability : you make a function that takes data and give data. You need to reuse it? Well you call that function. Done.

  7. about design patterns : these slides may help you understand that most of the one you know are just not needed



The point i have took time to think about before answering was your 1:1 1:* and *:* relationship. And i have a simple answer. 1:1 is a variable. 1:* is a variable that hold a sequence. *:* are.... a Dateclass or a function.

Finally you osgi link is 404.