3d design for printing

I don't want to sound like an idiot.  I really don't.  I just lack the patience to learn Blender.  It's not just because the name of the application sounds like a dating app for some subculture of swinger.  It is because it operates in a very similar manner to the Adobe Creative Suite.  There are menus for every button, and there are shortcuts that look very similar to finishing moves in mortal combat just to do something simple like replicate a part.  So, I had to put that down and walk away.

The sad part is that my main desktop system is feeling some neglect since I dork around with my pi systems and containers, but I do not actually investigate minor issues with my main box.  I didn't remove Totem until last week.  I just kind of lived with the jerky garbage video until I wanted to watch something clearly.  This is a problem because I could not consistently get FreeCAD to work.  So, for me, it seemed like I couldn't use the 2 most popular 3d tools to build stuff to print.  Then I spoke to a coworker.

A little back story about my history with 3d printing.  I asked around for recommendations on a 3d printer for a few weeks before pulling the trigger.  Everyone owned a different brand and didn't talk about it really at all.  Then I asked the guy who runs a print shop out of his house.  He owns 5 of the same model Prusa.  I think I pulled the trigger within 3 days of hearing about his print shop.  So, I asked him with help on design for a beer tap handle.  He immediately told me to check out Tinker CAD.  Within 6 hours I had a prototype.  Within 2 days I had a working model.  By the end of the week I had a working model that was in use.

The only real aggregate:
Tinkercad

Of course, once that was finished, I had to make more stuff.  The nice thing is that you can upload directly to Thingiverse from the app.  The instructions were very basic and allowed an easy path to get straight into working.  I like to print a few prototypes prior to finalizing a model, but it works so much better than printing ingots of plastic when using Blender.  So far I've made tap handles and smart watch charging stations.  Next up, the case for the pi IR blaster/universal remote from my first blog.  I'm pretty happy now.

I may have mentioned it in another blog post, but it is also pretty cool that you can print tools like a micrometer.  That helped quite a bit in the design of the charging stations.

Holiday and presents

Sometimes I need to remind myself that I am an adult.  I went ahead and made a deal with Mrs. Claus this past Christmas and got a Mavic drone.  It came with a half dozen accessories, a few weren't too obvious about their usage.  A little box labeled "propeller fixed" was a weird one.  I thought it was for propeller repair.  I had to look up images of proper usage, and it turns out that it is for holding the propellers in place when you put it in the travel bag.  A decent use of plastic.  The other piece was unlabeled and looked like the type of plastic that holds something in place during shipping, but came with an adhesive piece of foam to keep it from damaging anything.  Turns out that it is for protecting the remote for the drone when you pack it up.

The whole experience has been a reminder that I didn't do enough research and should have thought a little more about what I was getting.  The drone is great, works better than expected.  But, I had also been putting money aside to buy the racing goggles.  The options looked great and I thought I would need the Ocusync Air system for long distance connectivity, so I bought the system with the goggles.  Turns out, those are for adding a camera that is able to be used with the goggles to another drone.  So, I will be building an arduino drone, maybe with a pi to track people or recognize hazards. 

The real fun part about having the system is that now I have to learn how to use it.  The goggles are actually able to be used for full control.  They are also a bit of a pain in the ass because the entire right hand side is a touch surface that can enter menus and change modes through taps and swipes.  But, I can move the camera gimbal based on head movement now.  I will probably update again in the near future with pictures of the wrecks. 

Drawback of Open Source

I'm a huge fan of open source projects.  Most of the software that I install at home is open source, and I have thrown some money at a few projects that I like.  My own preference on the matter is that there should be contributions made by companies that use open source.  I've identified a few things that bother me about the current state of funding open source, some of which I will become very hyperbolic about.

RedHat.  The enterprise standard.  Please stop trying to charge for every single feature.  We used to make fun of Microsoft about their licensing.  How much is it going to cost for me to run a virtual container environment with software defined storage?  What if I want to have a local patch repository and centralized management? I suppose I will need to pay for RHEV, Ansible, Ceph, RHEL, Satellite, OpenShift, Docker EE, and whatever else you can find in there.  Or I can spend a few minutes looking up alternatives and use the cost savings as a selling point to the C level.  Convoluted licensing for products that were free up until they became RedHat products is a bad path to go down.

Also, notice that the Docker EE license comes from Docker?  That will not be the case soon.  Because RedHat couldn't figure out a decent strategy to make more money off of it, the are fleshing out Buildah as a replacement for Docker.  It also looks OpenShift is going to be a full step away from Kubernetes.  It's almost as if the community creates a standard, and RedHat makes enough minor changes that it no longer works with the community.  Don't believe me, take a look at what port they use for VXLAN compared to what is defined in the RFC. 

But, RedHat is the easiest way to pay for enterprise open source software.

Aggregate 1:
Ready to drop Docker in RHEL

Aggregate 2:
Standards being fleshed out in 2017

While I was scanning through the news one morning, I saw something relevant to my interests.  How to pay for open source projects.  This should be a no-brainer, just hit the donate tab on their website.  That's generally not how it works in corporate environments. The article sounded like a nightmare scenario for any admin.  Let another company scan your systems for open source software, and you pay the open source developers to keep your projects up to date.  Yeah, no thanks.  It is still a step in the right direction.  It does offer a method of payment from corporate customers, and it can make sure that critical software stays up to date.  But, if the dev team has heavily modified the software, then there is potential that it is not something that they want patched or changed at all.  It could be a part of a whole new application that the company does not want to rebuild every time some code change happens in the underlying technology. 

Aggregate 3:
A good start

So, the ideal solution in my mind is to find a way to pay for open source, reduce risk by holding the scanning tools, and being able to have a line item method of paying for mandatory components.  While I do not believe that Tidelift is 100% there, I think they are the closest to what is needed.  And, if enough customers agree on what is the correct approach, I'm sure Tidelift will create a policy based on that.  And no, I am not paid by them.  If you're looking to ditch some of the RedHat licensing and want to make sure that you can maintain support, I'd look into Tidelift.

Aggregate 4:
Pretty neat 

Yammering about 3d printing

I'm not going to pretend to be an artist.  I did get some toys so I can try, but I decided to start off much simpler.  My very first blog post was about building a home made IR blaster that integrates with AWS to make a voice controlled remote.  To completely finish that project, I tried to make a case for the project.  I now have a lovely ingot of PLA.  Not a big problem, it just reinforces that I have to learn more about the 3d design tools available.  Something I have not really had much incentive to do before now.  Which is kind of funny considering how often I seek out the design panels at different conferences.

About a decade ago I went to a conference that had a speaker connect his system to the projector and build out a skull in roughly 30 seconds, then drop it down some stairs by applying gravity to his design.  I think it was around the same time that the open source nerds were all freaking out about the release of Big Buck Bunny, a video made using only open source software.  They kind of missed the mark, making some of the creatures and themes too violent, dark, or deranged for small children.  I'm not trying to be as good as the 30 second skull guy or the Big Buck Bunny crew, but it would be nice if I could figure out the application well enough to make a box with a hole in it.

Adding injury to ignorance, I also tried a fun trick from my 3d printer manufacturer.  Soldering extra long pins to a pi zero w and adding it directly to the controller.  Within about a week I had realized that the system wasn't operating properly anymore.  I took the pi out and went back to a pi 3.  The system then degraded further and further until I found the root cause.  I had tangled a wire that led to the heated bed of the printer.  It caused a temperature failure for about 75% of my prints.  Once I resolved that, I realized that I could put the pi back into the controller box.  But I had a backlog of things I wanted to print, so it has to wait until after I get caught up. 

The most recent Chrome update has resulted in Octoprint being unable to preview the print with the GCode viewer.  I have found that it still works fine with Firefox.  I suspect that there is a memory leak somewhere, since my system will fail to come out of sleep mode when I leave Octoprint up on Firefox.  A minor annoyance since I use Chrome for everything.

I would have to say that my minor gripes are pretty funny.  I have to learn how to use an application that has been around for 2 decades.  I don't have time to upgrade my toy because I'm still playing with it.  I now have an increased workload because I have found more toys to make.  I have to use a different browser sometimes.  Meanwhile, Amazon and Prusa have created inexpensive filament that works well.  There are new attachments to make multi-color prints with dissolving supports.  I can make better toys for less money.  I just need to finish making my Halloween napkin rings so I can get started on Thanksgiving decorations.

High speed internet

Another upgrade in my neighborhood.  I got word about gigabit internet being available in my neighborhood and jumped at the chance.  I've noticed that while I have gigabit, it isn't as big of an upgrade as expected.  Downloading install disks for different operating systems still takes about as long as it did on 150 down.  What I did notice is that patching my own systems happens at lightning speed.  I think I was able to get a torrent of Ubuntu down in a few minutes.  Much like the old meme about coffee, I can do dumb things faster.  Pretty happy about it.

3d Printer

I've been slacking on updating for a few weeks.  I got a 3d printer.  After going through a few weeks of "Print all the things", I've settled back down.  I've had a few fun things to add about my printer, and the work I plan on doing with it.  Let me put you to sleep with my geek out.

I had a few discussions with coworkers and in forums about which printer to purchase.  There are a ton of options, all of which carry their own pros and cons.  One coworker owns 3 different printers, each are in the sub 500$ range.  One of them requires expensive proprietary filament.  Another coworker owns 4 of the exact same model that run 24/7 for an Etsy shop.  That sounded like the kind of machine I wanted.  I ended up buying a Prusa Mk3 that I put together myself.  It took roughly 8 hours to build it, and one part of the filament loading assembly snapped and had to be torched back together.  It did not break in a way that would risk a re-break from torching it, so I just rolled on, but be warned that it is a possibility to break a part when building.

The SD card that came with it had a bunch of stuff you can print immediately, so I went big with a dragon, then an offensive octopus, and eventually everyones favorite boat Benchy.  I connected to my desktop with a USB cable, then patched my desktop and rebooted.  That caused the printer to stop, which forced a reprint of the dragon.

I had some trouble with a few random things like boomerangs, and downloaded a ton of editing software and printing software.  There was one thing that confused me in all of this.  Prusa has a version of Slic3r made with their print settings already in there as a default.  You can also add a pi zero w to the controller on the printer to make it a wireless 3d printer with Octoprint.  But Octoprint can import your profile from Cura, not Slic3r.  Just a weird thing I noticed while messing with stuff. 

I also ran into a problem that I have stopped messing with.  Adding the pi to the controller.  I'm sure I could have resolved the issue with some electric tape and patience, but opening the controller box and dorking around in it took a few zip ties and 4 screws being removed and replaced every time I messed with it, so I decided not to risk messing anything up.  It seemed like the USB on the pi was coming in contact with the soldering of the heated bed, and I kept getting heated bed issues while the pi was in it.  Went with my new 3b pi and have been happy to give a full sized pi some purpose at home. 

Speaking of pi with purpose, I was able to print a case for my pi with the Matrix hat, finally.  And for my pi camera with IR bulbs, finally.  I made some wall mounts for echo dots, toys for my kids, and a really cool storm trooper pen holder for my tablet stylus.  Which leads me into another gripe.  Everyone wants to have the best tools they can get.  I got a Wacom tablet that is not in the kernel being released with Ubuntu 18.04, yet.  I messed around with Bluetooth settings to get support for my on-board card, but the tablet is only recognized when connected through USB.  Which makes designing parts fun.  It was described in another blog that using a mouse in Blender is like using a pencil that you are holding with chopsticks, so I really can't wait to be able to use my tablet.

There's a fun learning curve with the printer.  Clean it often, if things slip on the printing plate after cleaning use painters tape, some objects just won't print until you mess with them a lot in different software.  There's a not fun learning curve with Blender.  That application is up there with GIMP and Photoshop CS for functionality.  There are endless permutations to get the software to do what you want, which means that there are endless methods to screw up what you are creating.  I was trying to build my own case for the first thing I blogged about, the IR blaster.  I now have an ingot made of filament.  Practice makes perfect, so I will keep updating my progress on building my own stuff.  It can be very frustrating when you print Link from Legend of Zelda in such high quality and detail that you can see belt holes in a wrist strap, then you print your own work and it is basically a brick of melted plastic.  At least it was geometric. 

Your AI blog is old and my code is running on the wrong cores

Digging around into stuff like TensorFlow, you might come across some statements similar to "the idea has existed since the 80's, but the technology wasn't fast enough".  There are certainly drawings that can make that case, and probably some code examples.  What is frustrating right now is that the hardware is finally fast enough, and the code is maturing quickly.  That means that what was true two years ago is not true today. 

I am most certainly not blaming the bloggers on this.  I have a blog about creating an Alexa skill that can still be loosely followed today, but the Alexa skill website is completely different from when I wrote that blog.  I'm pretty sure that there was a significant difference in the video requirements for security cameras to the Amazon Show and Spot since I wrote my camera blogs.  You may not even need to create your own RTSP revers proxy in order to display them on your Fire TV anymore.

What I am having a problem with is the lack of decent blogs from the industry.  Hands down, the best industry blog I have ever read has been William Lam with VirtuallyGhetto.  Proof of knowledge and deep understanding from someone who responds to legitimate questions.  That is how the industry needs to do this.  Meanwhile, Amazon blogs about building cloud applications to cloud developers, but nothing for a tech savvy hobbyist. 

Aggregate 1:
Virtually Ghetto

I don't want to complain about the lack of knowledge coming from the industry, the only people qualified to write about it are much better versed on it.  I just wish they had some new guys that they could throw into the fire and distill the information.  Especially Amazon and Nvidia.  Amazon offers free cloud services that I use to turn on a TV in my house.  Nvidia is the back-end hardware for my AI containers.  So, when I see a blog that is incredibly well written from under a year ago, I trust the information a little too much.  But it was also one of the top results in Google search.

Aggregate 2:
Very well written blog

Unfortunately Nvidia has come out with version 2.  Some of the other containers in that blog are not up to date either.  They are from late 2017, and were probably put out as a single working release before the author/developer got too busy.  This is where the industry needs to step in.  The blog should be up to date from the company, the containers should be from the the company, and the instructions should be clear.  Considering how simple the instructions for install are, it is a disgraceful failure on their part that they do not have up to date documentation, especially for a company that charges for concurrent use of a GRID GPU on top of the 8k$ price tag to buy the damned thing.  They have the money and resources.

So, next blog I write will bhave the date listed at least twice, give specific instructions for a basic build on specific software, and will probably have multiple links back to Nvidia with offensive tags on the links.

The rapidly evolving hardware of SBC

I'm not really a hardware junky, I just play one in the planning meetings.  I was probably a little more excited than most people about the raspberry pi 3b+.  I had no idea what it meant, but there was a footnote on a few articles about power over ethernet, or PoE, capability.  This meant that I could add a pi and not have to consider power requirements to the location it served.  That's the most recent example of getting excited over a feature that I actually cannot use.

I've also experienced roughly 5 power outages in my house over the past month.  Not kidding.  I think I brought it up when I was bragging about buying my new computer hardware.  With what I have experienced at work, flash media dies quickly when you slam the power on it over and over.  I've had to recover time servers, single board systems, and some other random pieces of tech due to failed flash.  That's what makes the decision to get more flash based gadgets painful.  I need a good way to back them up, a load of spare flash media, and some use cases that don't require 5 9s of uptime.

So, with that in mind, I'd like to share the four devices that I will purchase based on my projects.  The first two are obvious, based on my blog.  The Pi and the Pi Zero W.  No need to dig around for numbers, the most recent are what I will use.  Standardized on micro SD, and for the most part they are also standardized on power.  Replacing a power block or drive requires little to no effort, and for things like security cameras, I have backups to clone from.

Aggregate 1:
Pi!

But, I am also prototyping a mechanical device.  That means I want a small form factor that responds to commands rather than a credit card system with a full OS that can replace a computer from the early 2000s.  Which is why I am looking into getting a few Arduino boards.  Commands and responses with cheap and easy Z spectrum add-ons.

Aggregate 2:
Arduino

And finally, I want a device that I can make a single use device for a specified purpose.  I mess around on my computer more often than not.  I recently forced an OS upgrade that stalled due to multiple previous upgrades, failed software experiments, and a massive hardware change.  I want to set up an AI system without worrying about what kind of strange stuff I am doing on my desktop.  And in a couple of months, I will be able to get what will basically be a chipset made just for that purpose.  It includes a CPU, GPU, and NPU (Neural-network Processing Unit).

Aggregate 3:
RockPro

So, who else has bought into features that they cannot use?  And who else is looking forward to tech that they have already planned a purpose for?

Modern computing, remote execution

I'm somewhat certain that the rise of secure remote execution was only a few years ago.  I've been playing around with some stuff at work that allows remote execution, but it is all locked into the same network, so it does not really answer my question.  When did we get the ability to securely execute remotely?  I'm sitting at my desk with a bag full of hobby electric motors and a few spare raspberry pi systems, knowing I can make these motors start spinning from anywhere in the world without exposing ports to the internet.  I'm probably going to use a combination of shell scripts, nodejs, and python to do it.  But this didn't just spring from the cloud.

I guess we will need to start from the beginning, the first computing systems that operated closer to a series of engineered gauges than a piece of hardware running software.  Let's try to follow some paths.  The very first path I want to look at is the Automatic Computing Engine (ACE).  This dives down to almost a century ago.  It was based off of classified work that Turing had performed, and may be the first view into how a program runs from memory.

Aggregate 1:
ACE

We will hit fast forward through some government contractor and academic nerd fights.  There was the Electronic Discrete Variable Automatic Computer (EDVAC) that used binary instead of decimal.  It helped lead to the Harvard Architecture that separated instruction and data in memory.  This is the first steps towards modern computers, but they lacked networking and the ability to run multiple applications simultaneously.  Probably the first example of using multiple computers for a single application was SAGE, using RADAR data to create a unified image.  This eventually led to ARPANET, which was the birth of the internet.

Aggregate 2:
EDVAC

Aggregate 3:
SAGE

This will eventually lead to distributed computing, which has some amazing characteristics.  Concurrency of components, lack of a global clock, and independent failure of components.  They distribute tasking, do not have high security, and have a low tolerance for failure.  Still, they were amazing when they were created.  This is what I had in high school and college on the internet.  Napster and online video games.  About 5 years prior, Object Request Broker (ORB) becomes a mainstream technology.  Issuing commands from one computer to another.  You can now issue commands from your desktop without having to login to the system that will execute the commands. Created without any form of trust, of course.  A long step away from our current cloud computing or domain methods.  These technologies will converge.

Aggregate 4:
Distributed Computing

Aggregate 5:
ORB

I need to pause for a second to apologize for the heavy reliance on Wikipedia.  The first part of this is history, the last part will be a little more cutting edge education.

Becoming more modern, we have Common Object Request Broker Architecture (CORBA).  Which allows ORB to work on Windows, Linux, Mac, Unix, and BSD.  You should be able to use any compliant language to issue commands to other systems. There is also The ACE ORB (TAO), but not the ACE mentioned in aggregate 1, a new ACE. We have gotten into the nested acronym portion of our evolution.  This ACE is the Adaptive Communication Environment, which is a framework that can tie together advanced features of operating systems.  TAO is a low level computing engine that can speak between systems to execute applications in real time.  When I say real time, in this instance it means "as if it were a local command".

Aggregate 6:
TAO

Aggregate 7:
ACE part 2

As security grew and became an integrated part of computing, some components of the model shifted.  Although the security was half-hearted, (security features were turned off by default in the OS) it was at least present.  Kerberos ruled environments with tokens and realms.  Massive SNMP configurations were used to trigger responses to events.  A serious mashup of security and automation.  I don't feel like linking to these technologies, and I have more to add to them.  There were LDAP and AD domains that could link into a realm.  SNMP was a shitty replacement for the old MQTT and DDS messaging that should have been configured in the first place.  But none of these things matter as much to consumers.  They wanted a way to use familiar tools to accomplish trivial tasks quickly.

Microsoft almost got us there with Simple Object Access Protocol (SOAP).  I still remember checking for SOAP port activity at an old job.  Messages that could control a networked application or device.  This is still heavily used today, and has been a great stepping stone to what I currently use for cloud based applications.  Representational State Transfer (REST).  While SOAP was built to be used on the internet, REST was built of the internet.  Commands on HTTP, with no need to open local firewall ports to execute fit the description of both SOAP and REST.  Simple messages that execute quickly, the ability to send code rather than a simple command, and floating on top of existing protocols rather than attempting to become a protocol are what make REST the winner. 

Aggregate 8:
REST

The nice thing about using HTTP to execute is that it is baked into every OS.  The uniform interface of REST makes it easier to tie in any other method of code execution that can be used.  Think about your smart home hubs, your lights may operate on a Zigbee spectrum using MQTT to send the message to each individual light bulb.  But the message that the hub is listening to is from REST.  For the time nerds, it's similar to Precision Time Protocol (PTP) compared to Network Time Protocol (NTP).  REST would be the NTP  that feeds the PTP that is MQTT.  While I have built many devices that use standard networks running on WiFi or Ethernet, converting them to an alternative spectrum like Zwave or Zigbee woule just be a matter of adding a hub to capture the REST command and sending it to the device with MQTT.

The future will obviously hold advances on the existing systems.  If we look at progress in a tick-tock fashion, I would guess that the remote execution has ticked.  The tock is machine learned behavior that executes without a command, being developed and fleshed out on the cutting edge right now.  The next tick will be an increase in speed and rapid reconfiguration based on integrated machine learning to further automate.  The next tock is basically what people will be gambling on the stock market about.  Could be micro-expression recognition to not only automate, but personalize based on observed response to an automation.  In ten years, your lights might turn down because your hub knows you are hung over. 

I adore comments of ridiculous speculation about the future.

Writers block cleared

I have a bad problem with writers block.  Not necessarily on my blog, I can yammer on about anything for at least 500-1000 words.  But when I'm writing a script or attempting to write an application, I can seize up pretty bad.  The typical problem is that I may know what I want the end result to be, but there are a few paths to get where I want to go.  I had been feeling especially locked up with how to play with one of my toys, an IoT hat for a Pi.  I just realized that I had been experiencing another creative block, not knowing what I might want to do now that I have something that I can use to start digging into creative uses of cloud technology.  So it sat there with me being able to look at what I can make the pretty lights do, checking temp and humidity, and considering what I might want to do with the facial recognition capabilities that are baked in.

My block is over.  I have figured out what kind of device I want to build, and now I am looking into what tools are at my disposal for the device.  I can't spoil the secret yet, but I think my goal is going to be building a proof of concept and then quickly selling out.  I will hopefully have a working model within 60 days, and then I can show off some videos of the device in action.  I guess if it stops being fun I will spill the idea and see if anyone can figure out how to build it.

Did you see this thing?

I've learned so much from people who start their conversation with "did you see this thing?".  The world is full of knowledge and knowledgeable people, but on occasion you can get a dud.  Someone trying to sell you on bad math or science.  There are massive gaps in all of our knowledge, and we need to be capable of self correction.  But, we also need to know how to share information and have information shared to us.

The key to being able to share really cool and new things is being able to understand them.  Since this is a tech blog, I figured I would share some of the better websites that have been shared with me.  I have had a nice older coworker that I refer to as my adopted Tiger dad that was helping me with some college math courses.  The first aggregate is pretty much the first thing he shared with me.  And yes, I would feel like I let him down if I got anything less than an A in my math courses.

Aggregate 1:
Best math site ever!

I was jumping back into math with Trigonometry.  A rougher than average high school led me to believe that it only had to do with triangles.  I honestly believed that it was going to be a ridiculous geometry class, until I started The Unit Circle.  That's a pretty special course on this website, because it purifies the math to a level where you could share knowledge with extra-terrestrials if you are capable of understanding the material.

Lets dig around a little deeper into my profession.  To get hands on experience with enterprise grade virtualization.  You want VMware knowledge.  VMware offers labs for free as Hands On Labs (HOL).  They do not offer a path to certification for free.  Let me repeat that so it is clear: VMWARE DOES NOT OFFER A PATH TO CERTIFICATION FOR FREE.  It costs a few thousand dollars, and it isn't necessarily worth it.  What you would be certified for is a technology that you could learn for free with their offerings.  Located at the next aggregate.

Aggregate 2:
My quick link to HOL

This was shared to me by the employees of VMware, probably a few days after it went live.  I still direct coworkers to it, making sure that they have access to the basics of how to work with virtualization.  I can't do anything about people not understanding the virtual stack.  I am still working on that with some basic white papers for my coworkers.

Everyone that has spent more than 10 minutes reading my blog knows that I have a burning desire to know as much as I can about Docker.  Infrastructure, development, commands, management, orchestration, and architecture.  I want to know all about it the same way I wanted to know all about virtualization.  Because I want to understand the common pieces of all containerization and jail technologies.  The more pieces and parts that show up in Docker, the more I wonder about where else they could have come from or be used.  Fortunately, Docker has been building out their knowledge base, and they now have a Hands-on-Lab.

Aggregate 3:
Docker HOL!

This is not going to be tuned to any particular vendor or OS.  So, my complaints about iptables on Photon OS are still just as valid.  Also, check out my blog about iptables on Photon OS if you need help understanding what could go wrong with iptables on a container host.

Finally, something I have not been able to dive into further than installing and playing a little, Machine Learning and Artificial Intelligence.  This is the cutting edge of computing right now.  In general, this is a big data technology.  You should probably have a whole lot of stuff to throw at it to make it a worthwhile investment of your time.  I have data I want to throw at it, I just don't have time to mess with it too much right now.  I also need to dig through the following aggregate.

Aggregate 4:
Making machines smarter

Some of my first blog posts were about setting up security cameras in your house.  Now, imagine being able to have those cameras identify who they see in their lenses.  Going a step further, imagine the security system knowing if that person should be there.  Getting a text message from a computer, with an image, asking if it is okay for someone to be in your house.  It may sound strange, but it is exactly what I am trying to build right now.  Stay tuned for a post in the near future about exploring facial recognition with Machine Learning and applying it to a system with AI.

The debate between whatever

This is going to be heavy on linking to tech opinion articles.  I have tried to keep my eye on tech trends for many years.  About 15 years ago I would pic up something like PC magazine and scan through it.  I still remember the first time I saw a WTF crazy article, it was written by John Dvorak.  To sum it up, he claimed that smartphones were bound to fail due to the high price and risk of losing them in a cab.  This was around the time of the Palm TREO, so he even got smart phones wrong again with his iPhone article years later.  His viewpoint is interesting, but he gets a whole bunch of things wrong about technology and trends.  For the most part, it is entertaining to read his articles.  The articles that I have seen popping up lately though, faulty logic and a disturbingly low understanding of tech and integration are on full display. 

Aggregate 1:
Dvoraks infamous iPhone article

Starting a few years ago, I noticed things like "the debate between virtualization and containers".  I had to take a step back and wonder what poorly constructed AI wrote this miserable article.  It was equal to a garbage post on a comment section.  The authors always seemed to be bringing up specific examples of how and why containers work better on platform X, which makes me wonder about their kickbacks.  Container hosts are generally going to be virtual machines.  Calling containers a VM killer is hyperbolic and does nothing to educate or enlighten the reader.  It is also ignoring all development trends with the biggest companies in both businesses.  I went about my business, creating VMs and not caring about containers until the tech was a little more mature.  It took about four years before I really started getting serious about using them in a professional environment.

Aggregate 2:
Containers vs Virtual Machines!!!

So, now that I am using them in a professional environment, the next big thing that we can quickly dispose of is "will the cloud kill your data center?".  These articles are always fun to read.  In some instances, yes, you will migrate to a nice secure cloud and your company probably will not save money when they do it.  But currently, I am not concerned about clouds destroying all data centers.  They are not as good at certain tasks and protocols that are requirements for a significant amount of companies, for now.  I'm not kidding, go sit down with AWS and ask how they handle multicast in their offerings.  There are also tools to migrate workloads to the cloud when necessary, once again making the technology a complimentary one.  So, while this is an actual threat to some data centers, it is not nearly the end of data centers around the world. 

Aggregate 3:
Cloud killed the on-prem star

My most recent example of ridiculous articles was based on a really cool offering from Amazon.  I have written about the use of AWS Lambda in my echo controlled remote article.  It is amazing technology and an excellent example of cloud services.  The sales pitch I got from AWS was that this is "serverless" technology.  A fancy way to say that they have isolated your code in an environment that does nothing but run scripts when necessary.  It has been referred to as "black box" technology because Amazon won't spill the beans on exactly how they do it.  It's a clever method to tie your code to a container running whatever language your script was written in.  This is where the term "microservice" becomes important. 

Your cloud based application, a container, probably has an engine that works on a specific long running task.  Serving a web page, monitoring a device, maybe a database engine.  But, for a specific event, there should be a script that runs.  In traditional computing, that script would run locally and you would have to worry about patching the application to run it.  Maybe that script offloads data to another application or service, which traditionally would reside on the same box or container.  With current cloud architecture, the correct flow for this would be Container A --> Lambda --> Container B.  You have broken a series of applications apart and linked them back together with a Lambda.  You can now scale horizontally to improve your application, which means you only need to turn on additional instances of whichever container is getting slammed.

This is obviously a complimentary technology.  I apologize for the long buildup on this, but I really wanted to frame this appropriately before I share the article.  Because, to me, this article is the equivalent of "RAM vs CPU, which should you put in your computer?".  To make matters worse, this is from a company that is trying to sell DevOps-as-a-Service.  It is their job to know about these things and explain them correctly. 

Aggregate 4:
containers-vs-serverless

No matter how much I want to write things like "John Dvorak is a crazy guy that shouldn't be taken seriously" after reading some of his articles, I am yet to find an instance in where he attempted to sell a service (other than as a commentator) based on his point of view.  I get it, people are trying to make money off of open source technology and vendor offerings.  They need to appear educated about what they are selling.  It doesn't help Caylent that they have massive errors on their "devops-as-a-service" sales page, and then present a testimonial from someone that is happy about how they set up his WordPress.  Not kidding, as of 06/15/18, they have the blurbs under "Built to Grow" and "Automated Disaster Recovery" mixed up.  This is a problem, one that involves credibility. 

Think about that whenever anyone tells you that some new technology is competing when it could actually be complimentary.

Why was this missing?

I've done a pretty good job of making amateur mistakes when I know better.  Modifying an xorg.conf file without making a backup is usually a favorite.  About 15 years ago I would rebuild the box because I didn't know any better.  I think I wrecked a file consistently for years trying to add additional mouse buttons to the configuration.  This time I did something ridiculously stupid.  My container hosts are mostly virtual machines.  I should be able to clone them in a few minutes.  Instead of doing that, I attempted to migrate all persistent storage to a NFS share.  As expected, it failed miserably.  Nothing important was lost, but it stung a little knowing that I had just wrecked something because I didn't even bother to look up the correct method of doing something.

I began my quest for information on the correct way to mount a share and use it for persistent storage.  Unless someone tells me otherwise, it looks like the correct method for a home lab is to install another package to allow NFS mounts in docker.  A package specifically for docker that is not included, and doesn't get the publicity it deserves.  I was quite surprised that it wasn't included on the host image. 

Aggregate 1:
Netshare plugin for docker

Of course the next step is figuring out how to use it.  Fortunately someone already wrote about that.

Aggregate 2:
NFS on a swarm

This is a pretty big deal to me.  There were some things missing in documentation that I figured out from searching around and tinkering, such as the port used for Swarm being blocked by iptables.  The absence of shared storage instructions may have been one of the reasons why my first attempt at creating a service and swarm was a failure.  I don't recall what all failed in that environment, but I do know that the ability to migrate containers with their own persistent data from one host to another should not be an after thought.  For something like the MotionEye container, the shared storage is considerably more important.  A simple NGINX config file is not hard to replace.  Security camera footage causing all containers to fail because the storage filled up is a problem.  That is exactly why I attempted to migrate everything from the local storage to a share.

It might just be the Photon hosts that I use.  So, coming soon, a blog about setting this up in Portainer and using it to create services to deploy to a swarm.

Add an APC (or other) UPS to Ubuntu 16.04 and Netdata

Netdata has become a method to push for actual statistical data on a system.  In my line of work, the actual statistical data of what a computer is doing while running an application is worth as much as the application.  That is the nature of computer science.  We have figured out creative methods of adding Netdata to running systems, including diskless nodes.  I have been running it at home for months, yet I didn't even look at the glaring issue of what was missing until I added it to my NAS.  I run FreeNAS at home, and you can look at their website to see my blurb about how much I like it.  It really does everything I want it to do.  It was ridiculously easy to add a battery backup and configure it in the UPS settings.  Then, I noticed that there was a service for Netdata included.  Turned it on, and now I can see the battery backup in a browser window when I look at Netdata.

The issue I ran into with my home lab box was not with Netdata, it was with the configuration and installed resources on my box.  Ubuntu 16.04 requires apcupsd to be installed.  I thought that it was automagic, since the system showed the battery in the status bar next to audio and language.  Nope, you need to install the package for battery backup.  So, I installed apcupsd and configured it. 

Aggregate 1:
Install apcupsd

Not sure if that will fix the problem alone.  I changed the UPSNAME to my UPS name.  I have a 1500 because power sucks at my house.  Modified the existing /etc/apcupsd/apcupsd.conf file as a copy rather than copy over the listed config.  The big secret to all of this is that I had to change the configuration of the status to allow it to be read.  I found out that I needed to change it from 0 to 1 in the config.

Aggregate 2:
Change the status

Next up is modifying the Netdata stuff.  Perform a locate to find out if you are changing all of the configs in the same place.  If not, copy and paste as necessary.  The key files you need to uncomment are listed in the Netdata configuration helper. 

Aggregate 3:
Netdata configuration helper

Then you just need to pay attention to the logs listed in /var/log/netdata just like grandpa had to. 

Shitcoin

I don't like bitcoin.  Never have, probably never will.  It is a speculative commodity that is based on nothing.  It is not currency.  It is damn near impossible to stretch the truth far enough to make it sound like a currency.  To add insult to injury, web browsers are having to incorporate measures to prevent miners from taking advantage of users that go to the wrong website.  That's right, you can have people mining on your box because you went to a blog.  I will do everything in my power to prevent that from happening on my blog, but mostly that is up to Google.  Even if you are a fan of crypto-currency, this is a problem.  You should not be mining for strangers.  So here are some things that help prevent it.

I found out about the latest and greatest from Firefox today, an extension that shuts down mining.  There are also extensions for Chrome, but they might be a little questionable.  The worst method of preventing mining is also the easiest, turn off java script.  Then you can't use any websites.

Aggregate 1:
Kill it on firefox

Aggregate 2:
The #1 on Chrome

Any method you choose to use, do it.  Allowing people to mine on your system is ridiculous.  If you want to do it, that's your choice.  Don't let people waste your electricity for free.

EDIT: Adding onto this due to breaking news over the past week.  There were contaminated Docker images on Dockerhub.  Surprise, crypto mining software was inside the containers.  I am starting to become entertained by this.  I get it, everyone wants a get rich quick scheme.  The funny thing is that now people are trying to get you to unknowingly run code that does not destroy your files in order to make money.  It's probably the best case scenario for malware.  It's not even ransom ware.  While it is still annoying, and a wake up call to make sure you are pulling from good sources/reviewing what is in the container, it could be so much worse.

Obligatory ML/AI blog

I was told by my NVIDIA rep that I am on the right path and at the very beginning of it.  I'm trying to set up some basic AI dev environments for people to screw around on and start thinking about what they want to focus on if they decide to create a project.  Enterprise grade AI dev environments are really big compared to what you might have at home.  This isn't a single user with a graphic card running TensorFlow.  Nope, it will be about 5 users with graphic cards running TensorFlow.  I am using it as a method to determine demand before I go and grab whatever purpose built chipsets I can get my hands on.  Since I am looking into new methods to support potential customers, I figured I should share my very basic research into AI.

First off, I'm using nvidia-docker.  This decision was made lightly, because it provides the portability and resource configuration capabilities that I need.  Also, I am using a 1070 in my system at home.  Second, I am using TensorFlow.  The other major offerings come from companies like Facebook, which don't deserve the leniency that Google has earned after years of not being offensively obvious about data mining.  There are more blogs about setting it up than I can link, but I will share a few things with you about this so I can complain on a public forum.  Not many instruction sets include merging your docker config file and there have not been many updates recently (as of this writing).  Let's get to it.  Nvidia-docker install with a merge to make it work!

Aggregate 1:
The basic install instructions

Aggregate 2:
Adding the NVIDIA runtime to docker

After these steps are followed, in a bit of an aggregate 1-2-1 kind of way to make nvidia-docker work, you should have a working copy of TensorFlow.  The annoyance after that was Chrome becoming unresponsive.  I have netdata running on my box, and there was absolutely nothing out of the ordinary the first few times I ran TensorFlow and had it crash my browser.  I then opened a new Chrome window exclusive to Jupyter notebook and everything ran fine.  One thing stuck out, and it was mildly hilarious.  TensorFlow complained when the amount of available RAM dropped below 4GB.  It was in the shell that I had executed from initially, but I looked at it wondering how often it would show up in the logs when placed into a dev environment.  Not to sound like a crusty old admin, but 4GB is a lot of RAM.

Once the Jupyter notebook was accessible based on the instructions, I went ahead and ran all of the sample notebooks.  Running them in their own window earned me the ability to complete all notebooks without having chrome lock up.  At the end of notebook 3, they mention that you can run one of the last sets multiple times to increase accuracy.  It does do that, but it will take some time.  My graphic card chews through it in under 5 seconds, but validation error decreases by about .1% after every 5-10 runs once you hit the 1.5% test error threshold.  Next steps, figuring out how to export what the system learned.  Not a good problem to have, considering that many users can't get their systems to complete the sample notebooks.  That means documentation goes down by about 90%, and I may not have much to share that isn't behind a pay wall. 

What I have learned from the obligatory AI test is that there is still a whole branch of computing that requires support and security that may be overlooked for the time being.  But it is coming, and we should all push for this to become a highlight with the admins.  Typical AI uses revolve around big data.  We should also focus on data security regarding input and output to ensure that we do not process bad data or present bad data.  Hardware requirements and messages/errors need to be able to be tuned so that we can capture and configure how the data is processed and determine if there are problems.  These issues are form fitted to the environment, so it will take some effort to determine what is appropriate.

After roughly 20 tests, I have a 0.8% error rate in determining what a hand drawn number is.  I will blog again soon about how to push that knowledge outside of the AI container.

Migrating from SSD to M.2 or NVMe in Ubuntu

A couple years ago I started working with NVMe drives at work.  They advertise "up to 6x the speed of SSD", and are pretty neat to play with when working with vSAN.  Of course these are industrial grade NVMe drives, which are pretty much a board with a bunch of M.2 drives on it and a string of capacitors with some other chips sprinkled in for advanced capabilities.  I obviously needed to have something similar in my computer.  So I ordered a 960 EVO and PCI-e adapter.  First impression is that these things are small.  About the size of laptop RAM.  My next observation was that my old box wasn't going to be able to use it as a boot drive, it was just too old.  So I put it in my system to use for ripping and transcoding video files.

Aggregate 1:
Isn't it pretty?

But then I upgraded to a motherboard that has an M.2 socket, so I had to make it my boot drive.  The nice thing about Linux is that it tends to be easier to make backups and move the OS around.  So, I built my new box, popped in the old SSD, booted up and made sure everything worked the way I wanted it to.  Then, I had to study to migrate the data, and find the missing pieces in the instructions.  First step was to create a Ubuntu DVD, I used 18.04 since I knew it would have a decent nouveau driver (got a 1070 in my new rig).  Booted from the disk and ran it live, which takes a long time to complete boot and get to a working screen.  This part was rough because I had to do it a few times due to missing instructions. 

Aggregate 2:
The Bionic Beaver

Once booted, I needed to install gddrescue and copy the SSD to the M.2.  I probably could have figured out how to do this with dd, but I was hoping this would be the one and only step.  For those that intend on pulling out their old SSD after upgrade, it probably could be.  A quick note before anyone begins on this endeavor, check the block size of your disks.  I got lucky with uniform 4k blocks on my devices, but I have seen 512 and 8k block sizes on devices.  A simple blockdev --getbsz /dev/sda1 to view block sizes on that partition can mean the difference between completing the task quickly and having to reboot into a live disk an additional time.  The instructions for using ddrescue are simple enough,   sudo ddrescue -v --force /dev/sda /dev/sdb to clone data from one drive to another.  NVMe or M.2 would look a little more like sudo ddrescue -v --force /dev/sda /dev/nvme0n1  and the full instructions for moving stuff around on bigger and smaller drives are in the following aggregate.

Aggregate 3:
Migrating the OS

The part that I got stuck on was the UUID.  Cloning the drive leaves the new device with the identical UUID of the old one.  I had to make changes to the drive, so I decided to fix it in gparted.  Yes, this can also be done in parted, if you want to get down like that.  The first task was to edit the M.2 drive to use all of the storage, since it has double the capacity of the old drive.  This part is dangerous, you can break the install on the drive you are modifying if something goes wrong.  That is also the reason for the UUID change, so I could easily get back to a working SSD if something bad happened.  Once the capacity change is completed, go to the Partition menu in gparted and give each partition a new UUID. 

Aggregate 4:
Gparted capacity increase instructions

Now that you have new UUIDs associated with the partitions, you need to fix your boot parameters.  What was normally a difficult and boring part of Linux administration is automated, which is great because there are too many ways to fat finger a UUID if you have to enter it manually.  All that is needed is to run boot-repair.  Just make sure you check out all of the options, select the drive you want to repair (nvme0), and only write to that drive.  That way you can pull the M.2 drive out and go back to the SSD if things seem to become too dire after reboot. 

Aggregate 5:
Boot repair, but not from a cobbler

Reboot your system, verify the boot option in the BIOS, and see how fast the system comes to life.  I've heard some optane systems can be up and running before you pull your hand back from pushing the power button.  Once you are sure everything is to your liking, you can wipe the old drive and use it for tier 2 storage, or a new mount point for /var.

New Hardware (filler)

This is a complete fluff post about my new hardware.  I just got a new Ryzen 7 system with 32 GB of RAM and a Nvidia 1070 card.  I will have some posts about nvidia-docker and AI in the very near future.  I also got a new battery backup, because all of my old UPS systems were trashed, which helped lead to me getting a new system.  So guys, if you need a new system and need approval from a significant other, stop replacing your batteries in the battery backup units.  It only took my system a few years to die when I did that.  RIP 6 year old Ebay used parts special.  Welcome back modern computing.

On an interesting note, I only had to move the hard drive from old hardware to new hardware, then modify some network and display settings (I got new monitors too).  Very simple and easy transition to new hardware.

Bias Lighting and USB

My family is pretty brutal on the television binge sessions.  A new show will come out on a streaming service and we cancel our weekend plans to watch it.  Not a great strategy for shows like "Black Mirror", since you need a day to digest the episode completely, and maybe a trip to a therapist.  One of the items I kept seeing in smart home social media threads for binge viewing was bias lighting.  It's a neat little hardware add-on for your television that prevents eye strain and assists with contrast.  Grays seem grayer, blacks become blacker.  A neat concept, so I did some more digging.

One of the first things that came to mind was the lingering expectation from Phillips to integrate the Hue color bulbs and LED strips into your smart TV or Roku/Firestick/AppleTV/Chromecast.  It's been referred to as "the immersive viewing experience" on the This Week in Tech (TWiT) podcast a few times.  That is not what bias lighting is.  Bias lighting is an attempt to match the temperature of the lights that illuminate the screen at a reduced lumen by reflecting and diffusing the light source.  That's a pretty direct way of saying that the light is going to be as cold as the florescent tubes that light up mental institutions, but it will be behind something so it won't be as bright, and it helps protect your eyes.

Aggregate 1:
What is bias lighting? 

I added the LED strips to my Amazon shopping cart and went to sell the idea to SWMBO.  After about 45 seconds I realized that she knew more about it than I did, and had probably written a paper on why it should be an OSHA standard to have bias lighting on all computer monitors.  She was already sold, so all I had to do was hit the checkout button.  Here's where things become a little "first world problem".  They sell inexpensive stick-on kits, which are great because I love to save money.  The problem is that it is a one size fits all for a range of sizes.  One that fits a 55 inch TV can also support a 75 inch TV.  You can buy specific lights for your television brand and model, for a price.  But I am unwilling to pay a 300-500% markup to have one that is specific to my TV, which probably requires taking a mounted television down so it can be gently placed on, and the adhesive is then given hours to cure.  So I got the one size fits all.

Aggregate 2:
Extra large LED strip

I went ahead and wrapped my largest TV, with no excess LED light strip to deal with.  Looks great, just have to adjust the angle to prevent a very close and direct reflection of the LEDs on the bottom.  The plug is a cute little integrated USB that plugs straight into the screen and turns on as soon as the TV does.  Next, onto my smaller TV.  I ended up with about 4 feet of excess, and I was able to easily cut it off at the "cut here" mark.  I'm one of those people who grew up with an education of the environmentalist 3 R's: Reduce, Reuse, Recycle.  4 feet of copper and LEDs surrounded by plastic shouldn't end up in a landfill, it's wasteful.  So, I did what any geek with a soldering iron would do.  I made another bias light.

Noted in aggregate 1 is the ability to use a single strip in the center of a TV that is not mounted.  One of my kids has an unmounted TV that would be able to make use of the excess.  The trick was to connect it to USB so that it could be plugged in and used just like the others.  The pinout of USB 1 and 2 is dead simple.  You have red (5 volt power), black (ground), green (data in), and white (data out).  We are interested in the red and black.

Aggregate 3:
USB pinout

Caution: I believe certain USB 3 and C cables and interfaces can reach higher than 5 volts without making a data connection, but don't quote me on that.  Verify with the cable manufacturer and do as much homework as necessary if you need to use those cables.  

To connect the USB to LED, simply solder on the red to 5v+ and black to 5v-.  I recommend cutting the USB cable at around 6-12 inches, whatever allows the least amount of bend and doesn't protrude out from the side when plugged in.  The remaining portion of the USB cable can be saved for experiments with serial and network connections or as a replacement for a damaged cable.  Worst case scenario, find an electronics recycling facility or event.  To solder, tin the pad and wire, then solder them together.

Aggregate 4:
Solder wire to board

Once you have soldered everything together, pull the adhesive protection on the LED strip down enough to wrap about an inch of the LED strip with electric tape.  Wrap from an inch above the solder to an inch below where the USB cable was cut open.  Plug it in to verify that everything lights up, I recommend an appropriate USB power adapter that is plugged into a socket that won't take down all of your power if you have a short circuit.  I'd show pictures of the final product, but I'd get yelled at by a 12 year old about how I wrecked his game of Fortnight.  The luminoodle is tuned for USB, and is ready for this project.  I'm so glad I didn't have to add tiny surface mounted garbage to make this work.

Note:  Most TVs are USB 2, which can be determined by the color of the USB interface on the TV.  White is 1, black is 2, blue is 3, red is the newfangled version referred to as C (as of this post).   Make sure you verify and test with each type prior to leaving it plugged in without supervision.  Don't burn down your house because you refuse to test something you built out of spare parts. 

Backslash or no Backslash?

One of the minor annoyances that I encounter regularly is a common one on the command line.  People who primarily use Windows tend to be big offenders, mostly because you don't tend to have the same issues in Windows as you do elsewhere.  It's the folder and file name with spaces issue.  Most of the Linux professionals I know use an underscore in place of a space, which helps with scripts and config files.  Windows users tend to name things with spaces in the file or folder name, because it generally doesn't matter in their shell or batch files.  It still messes me up when looking for something and I forget to add the backslash to get to it.  An example of what it looks like:

Windows  C:\Documents and Settings\User\Documents\I bet you hate spaces
Linux  /home/User/I\ bet\ you\ hate\ spaces

It's really not a huge deal, but it has become ingrained in my head as the correct method to deal with spaces in config files and such.  But the GUI doesn't care about how we do it in the shell.

I had decided to configure Tautulli to look a little deeper into my Plex server, and get a good idea of system usage.  This ended up being a little bit of an ordeal for me, since I have kept Plex in a jail on my NAS since I built the NAS.  That meant that I had to add a folder on the NAS to share to the jail, migrate the log data and mount point from the jail to the new storage, and then point to the new storage from my Tautulli container.  Somewhere in there I also had to mess with users and groups to make sure everything went smoothly.  So, after about an hour of messing with stuff, I was ready.

Aggregate 1:
A nice addition to Plex

Aggregate 2:
Tautulli Container!

Used the same configuration info on the Tautulli Docker page to set up the container in Portainer, and received the error that it could not find the log mount point.  I started thinking of ways to check since it wasn't really getting far enough to generate anything useful in the logs.  So, after about 10-15 minutes, I decided to remove the backslashes and see what happens.  Of course that worked. Which was annoying since the timezone entry had to have capital letters and used an underscore, just like a normal shell command.

I guess the lesson of this blog is to remember that sometimes there is a difference configuring things in a GUI and a shell.  And now I can see who is watching my server.