These forums have been archived and are now read-only.

The new forums are live and can be found at https://forums.eveonline.com/

EVE Technology Lab

 
  • Topic is locked indefinitely.
 

ORM vs straight SQL for Data Dump

Author
Dardo
Caldari Provisions
Caldari State
#1 - 2012-03-08 17:44:29 UTC
Want to get everyone's ideas on whether or not it's worth it to use an ORM for the data dump.

I'm finding since there are so many join tables, it's almost not worth using an ORM and just using straight SQL.

Anyone else have thoughts?
Steve Ronuken
Fuzzwork Enterprises
Vote Steve Ronuken for CSM
#2 - 2012-03-08 18:01:33 UTC
/me shudders at the mention of an ORM.

From a DBAs perspecitive, ORMs are pretty much the devil. They divorce people from the technology they're actually using, so the ORM ends up writing really bad sql to use the underlying tables.

And they don't think about the indexes they need to create, because the ORM hides it all.

Sure, it makes things faster development wise, which saves money. but I don't have to like them.


Anyway, in this instance, I guess it all depends on what you're going to do with the data. I /suspect/ you're going to be writing reports with it, rather than manipulating it. Which suggests, to me, using regular SQL.

Woo! CSM XI!

Fuzzwork Enterprises

Twitter: @fuzzysteve on Twitter

Dardo
Caldari Provisions
Caldari State
#3 - 2012-03-08 18:03:54 UTC
Agree with the DBA perspective, but in this case, the DBA would be...me...lol

But I'm actually thinking from a development perspective, SQL might actually be faster anyway. Rather than setting up complicated joins on the entities to get the information I need, just write some SQL for specific things I need.
Callean Drevus
Perkone
Caldari State
#4 - 2012-03-09 14:51:53 UTC
I think the point of ORM, at least Entity Framework, is to have you do as little joins as possible, but just look up everything when you need it. At least, as long as you use deferred instead of eager loading (which would probably load half the DB with the SDD) Blink

Developer/Creator of EVE Marketeer

Dardo
Caldari Provisions
Caldari State
#5 - 2012-03-09 16:35:38 UTC
ORMs definitely lend themselves well to document databases, or NOSQL.

Databases with tons of joins such as EVEs db, makes it extremely annoying since you have to explicitly define the joins, rather than letting the ORM create the join tables for you which it normally does.

So yeah, I'll be using standard SQL.
Etil DeLaFuente
Aliastra
Gallente Federation
#6 - 2012-03-10 15:24:49 UTC  |  Edited by: Etil DeLaFuente
Using an ORM is all about having an abstraction layer hidding your persistence layer. It will help you migrate from one solution to another, just by modifying couple of configuration files. NHibernate, for instance, can handle a wide range of solutions.

ORMs provide change tracking capabilities, transactions etc... all the kind of stuff you will have to implement yourself and maintain !

As for the DBA perspective, you will have to test the generated scripts and create indexes where it matters. Also use views and map flat objects to them if reading performance is important to you.

S.O.L.I.D !
Dardo
Caldari Provisions
Caldari State
#7 - 2012-03-10 17:04:53 UTC
Good perspective.

I'd be using Java Spring, can get all the transaction management easily through the use of annotations. I do agree with ORMs allowing you to move between datasources easily. Especially since I'm thinking about pushing my app to Heroku which I believe uses Postgresql, rather than MySQL which I'm using locally to test.
Ilyk Halibut
Deep Core Mining Inc.
Caldari State
#8 - 2012-03-11 05:50:25 UTC
Anyone posting in absolutes about ORMs is showing real ignorance. It largely depends on usage case, and the ORM in question.

There are lots of different approaches and implementations, and not all are created equally. Most ORMs also give you a way to drop to raw SQL if you just can't make it work for certain queries. Depending on what you're doing, an ORM can save you a lot of time, help protect you against injection attacks, and make your code a lot more readable. Of course, they can also lead to SQL-soup behind the scenes, but that's something you have to keep an eye on.

Since I assume we're talking about a hobby project here, the main things are probably productivity, and "fun". So do what is more fun, because historically, you're going to run your project for at most, a few years, burn out, then close shop. The further you can get before losing steam, the fonder you'll look back on your effort :)

EVE Market Data Relay - A real-time feed of EVE Market data http://www.eve-emdr.com

Dardo
Caldari Provisions
Caldari State
#9 - 2012-03-11 05:56:19 UTC  |  Edited by: Dardo
Agree 100%

Unfortunately, the EVE tables don't lend themselves too nicely to ORMs, imo considering most of them are join tables, with FK relations, and nested parent / child relations (see invMarketCategories)

Why I'm thinking about using straight SQL.

Not too worried about SQL injection since I'd be using named parameters, etc.

Should be fun, nice to see people so responsive =)

if you want to take a look at what I have thus far:

https://github.com/donkeystalk/eve-backend

have a small frontend project too, but might combine the two since no real use to two projects. Going to use backbone js and hopefully have it be a single page app, like you alluded to, this is a pet project =)

https://github.com/donkeystalk/eve-frontend
Wind Jammer
Molden Heath Software Company
#10 - 2012-03-11 11:00:04 UTC
I'm a programmer not a DBA, so my main interest is saving on typing and reducing boilerplate code. ORM is one way of achieving this, but it can be a bit of pain to set up. Also, quite normalized databases like the SDD will usually need another transforming layer to put your data into more useable domain classes.

I also use plain SQL through Spring's DAO and JDBCTemplate which is fab for not needing having to worry about irritating exception handling code. It's a bit less reusable than the JPA classes, but it's so easy to sling together another query I'm not bothered.

So I'm using both, whichever suits me at the time.

For you DBAs, ORMs aren't necessarily more evil than writing SQL. Just like hand-writing SQL, you still need to tailor and tune it, just in a different way.


Dardo
Caldari Provisions
Caldari State
#11 - 2012-03-12 00:06:50 UTC
Right, all I use at work is jdbc template for the most part.