Saturday night fun

Lon dev's life == lots of joy!



    # software development is very recreational
    self.recreate_client()

Sadly this bit might be being deleted.

Pandas +1 thing to be nagged by

ipdb> df = pd.DataFrame(dict(x=[1, 2, 2, 3, 4, 5])); df.set_index(["x"], inplace=True); df.index.get_loc(3, method="pad")                    
*** pandas.core.indexes.base.InvalidIndexError: Reindexing only valid with uniquely valued Index objects

Without method="pad" things seem to work. Not sure I should flag this up, though the error message is at least a bit misleading. Then not sure if others find newer Pandas versions on some channel, mine is a 0.24.2, there are docs for 0.25.2 out there. Bug? Feature? I guess it's best termed as 'room for a feature'.

Yaaaay it's Mondaaaay :)

Then not sure if I'm completely right to just get around the problem using np.searchsorted() with roughly the desired effect, but in the given case "behaviour exactly as thought out" is not that indispensable. So I think I just will. Shame on me.

Sure an anti-pattern? "Asking for forgiveness instead of permission"

Among Python's idiomatic approaches there is "It's easier to ask forgiveness than it is to get permission".

It may be a brilliant thought in context, but could possibly turn into a silver bullet when torn out of its habitat and forced into an over-generalized application. See, that isn't always a "nail" for your hammer.

What I mean is let's say I copycat a prime example mentioned here.
(Google cache link)

First of all, as of now the anti-pattern example is

Versus the recommended:

Now exception handling isn't difficult to get right, but that doesn't always happen anyway for whatever reason.
The recommendation is slightly (?) incorrect. The two implementations aren't equivalent at all.

The "pop-pattern" (popularized pattern) catches (and allows) a range of exceptions to take place. There probably isn't a difference made between say hard drive failures, read-only and non-existent files.

Of course if the file was read-only (from the perspective of the code at least, e.g. due to insufficient rights), whilst the rest of the implementation assumes it is deleted, then further unexpected situations are likely to arise, reducing the robustness of the software.

So there's the situation: we're misinterpreting a new exceptional case, and it may in the end just never arise in production, because the exception is not communicated. And what worse than a software defect that is not noticed ...

Or let's say you catch that very specific exception that is triggered when the file to be deleted does not exist. How can you be 100% certain it was the non-existence of that very file that triggered the exception? Yes, it's plausible to assume that the unlink (remove) code will not involve other files, but generally ... no, the code blocks still would not be equivalent.

This isn't my point, but I believe it points in the right direction.

The "pop-pattern" may make maintenance difficult

First of all, in real life these exceptions often are noticed in practice. "Ooops" says the programmer and adds a try..except block. Or better, in expectation, but that might require the extensive analysis of the documentation sometimes. That which might just not happen sometimes. Yes, people aren't perfect (and hopefully don't even try to claim to be).

So then the code may change slightly. Say from os.unlink to "remove_file(...)" in your own implementation, dealing with local/remote storages, whatever. You'll surely have to watch out, as the origin of the exception starts get farther and farther away from the handling code, as complexity in the exception handled code sneakily builds, your exception handler will be increasingly likely to get confused.

Had you used a particular error code - that would expressively restrict the code maintainers to provision for you as told. Well, hopefully. Shortly, my impression is exception handling can be more costly in the long run than to watch out for the individual situations, which are not necessarily clearly communicated in the forms of exceptions, and even if they are, the respective checks are counter-intuitive to make. I wonder how often people are seen actually testing for that filename member of the FileNotFoundError exception :)
(Ray of hope: there was at least one guy :) but a GitHub search suggests that it isn't the norm - and frankly, who on Earth has time for this?! outside this too long train of thoughts)

The fun bit

Quoting from the Python 3.8 documentation:
(Google cache link)

os.unlink(path, *, dir_fd=None)

Remove (delete) the file path. This function is semantically identical to remove(); the unlink name is its traditional Unix name. Please see the documentation for remove() for further information.

New in version 3.3: The dir_fd parameter.

Changed in version 3.6: Accepts a path-like object.

os.remove(path, *, dir_fd=None)

Remove (delete) the file path. If path is a directory, an IsADirectoryError is raised. Use rmdir() to remove directories.

This function can support paths relative to directory descriptors.

On Windows, attempting to remove a file that is in use causes an exception to be raised; on Unix, the directory entry is removed but the storage allocated to the file is not made available until the original file is no longer in use.

This function is semantically identical to unlink().

New in version 3.3: The dir_fd argument.

Changed in version 3.6: Accepts a path-like object.

Briefly speaking: the exception that is caught in the example code block is not featured in the official docs for laymen. In other words, there is a potential for this exemplifying an "oops" said the programmer situation... in the given case it's probably the right exception to catch.

But (don't ever) trust me (but still give it a thought that :) ) you wouldn't like to always operate in this  trial-error fashion when provisioning code likely for the long run, and possibly for a sizable user base.

Conclusion

I'm not sure there's one :) But I'll try to come up with something
In my opinion this particular one is a dodgy anti-pattern: a good thought, but very easy to misinterpret.
Popularly mentioned anti-patterns should still be avoided with care. In my experience, exception handling should be as specific as possible (and I'm not sure if exception handling can be specific unless the framework the code relies on is restricted to be a specific version and/or specification), or bad things can happen.
Don't overcatch, don't overapply :)

Things to ha... strongly dislike in Jupyter

Well, come on ... so by default the plots are aligned to the left.

This is so, despite images seem to be centered, which is - from a layman's perspective - 'well, odd'.

It's not the end of the day, you can adjust it ... but wait, seriously? You can adjust the plot sizes by matplotlib whatevers, but not the alignment?

I guess at the very least nobody should be dreaming of orthogonality here :)


https://stackoverflow.com/questions/18380168/center-output-plots-in-the-notebook


The downside is that "I googled it" hacks are going to form the basis of the development culture.

Well, maybe a feature request helps, not sure, but having to use HTML & CSS for something that should just take place by default and is probably one of the most commonly experienced things is a bit more complicated than it should be.

eToro in a short post

Barry Jones

>>> x = [1.09, 1.84, 15.04, 3.61, 0.68, 7.62, 16.60, 15.16, -6.53]

>>> (np.exp(np.log(np.array(x) / 100 + 1).sum())) * 10000

16687.288633717548


vs.

saying

27038

Err.... what the ... lol?

I guess it is time to say goodbye to copy trading...

I mean,
a) can't eToro just come up with 2 measures in 12 years that accord with each other?
and
b)
(i) not detailed here, but costs probably make performance measures misleading (in my experience)
or if not (i) then
(ii) when copied users add funds, that makes it difficult to interpret their performance, and furthermore
c) while copied users get rewarded for their success from their followers gain (which reward can by the way be reinvested), will they suffer proportionate losses when their copiers incur losses, so as to increase their awareness of the risks they are undertaking? I would easily take a guess (i.e.: no). So - they are basically incentivised to take risks, being under penalised for their failures.
Smells like negative selection, my friend.

So, great idea, but the execution leads to a likely early goodbye, even though this guy's the current best performer (and got there very quicky!) in my virtual portfolio.

By the way, my copy portfolio is as:

All of these folks were editor's picks at some point.
Barry Jones is the only recent addition, the rest had at least a month or two to show off.
Clueless how to check when I actually started copying them.
12 years of development.
Well done, guys.