People sometimes write code that is very assertive about this or that piece of a system. In a more suspicious case, many parts.
This typically isn't going to yield performant code - something will often try to slow your job down.
A specific advise:
0) Keep (simple) processing close to the database.
What I mean by simple is something that easily could be translated to a few machine level instructions (e.g.: sum) in each iteration, is done in a few iterations (e.g. 1), doesn't operate on a large dataset in each iteration.
Let's generalize it:
1) Keep avoiding to cross boundaries between system parts unnecessarily.
Or: try to consider alternatives where fewer communication routes are involved.
Such means of system boundary crossing to be avoid could be anything really, depending on the scale of operation, and at what magnification we consider the system:
- Querying data from a server
- Calling into a DLL/library
- Exchanging information with a service/daemon (inter-process - but also, probably with a smaller overhead, synchronized inter-thread communication).
- Looking up data from another table (one implication: look for unnecessary JOINs, maybe consider denormalization, esp. in a NoSQL setting)
- Calling Java bytecode from a compiled binary
- Calling another function (remember there's a cost of leaving return information on top of the stack)
- Blocking on-screen confirmation with the user
A major exception: if the overall complexity is high.
In this case it may be worth turning to some hard/software dedicated to the processing - using a
GPU, an SSD, scaling up, a cluster etc. a few terms to think of as
opposed to CPU/FPU, HDD, your regular hardware, single compute node. And this will all involve moving data from one part of the system, less suited for the processing task, to another, more feasible, possibly more dedicated, potentially a temporarily allocated resource from a cloud.
However, it's also worth noting:
faulty and or critical system components may cry for redundancy
Such as - people. And then there come the desirable boundaries - but also come peer insight. Whether the superior efficiency of the one-man teams is a myth... well, while we'd like to believe in myths, we do know, that may not always be the best that can happen. You may need very proficient and disciplined people for that to work out - these days, with an expanding IT industry, years of experience on average is going plummeting.
Stumbling upon them could be way more the exception than the norm.
