108 replies to this topic

[mark77]

It was suggested that I start a new thread about what I have been working on with the Raspberry Pi.

 

First, I need to define what ALPACA is.

Alpaca is a new protocol that was released at NEAF 2019.  It was designed by Bob Denny, the same guy that designed the original ASCOM prtocol.

 

Alpaca is "ASCOM for the rest of the world"  It is a network based protocol using JSON and RESTFULL interface at the lower levels.  It implements the same functionality as ASCOM but across a network and it can run on ANY platform, not just Windows.  It is compatible with ASCOM if  you install the ASCOM/Alpaca bridge on your Windows machine that is running ASCOM.

 

 

Ok, now on to the Raspberry Pi.

Since April 2019, I have written 73,000 lines of C++ code to implement the alpaca protocol.  I have followed their rules and guidelines but I have also embellished here and there and added more functionality.  I will get to that later.

 

I have written both drivers and GUI applications to talk to those drivers, all on Linux.  Eventually I plan on making it open source, but I want to make sure everything passes the ASCOM/ALPACA test suite before I release anything.

 

IMPORTANT NOTE: Alpaca DOES NOT REQUIRE ASCOM and DOES NOT REQUIRE WINDOWS.  This was very important to me because I prefer linux over Windows.

 

 

I have implemented the following

Camera

      ZWO

      ATIK

      QHY (mostly done)

      ToupTech (started)

      FLIR

 

Focusers

      Moonlite NiteCrawler

      Moonlite HiRes

 

Filter wheel

      ZWO

 

Rotator

     Moonlite NiteCrawler

 

Dome

    My own dome using R-Pi relay board and motor control

 

Switch

    R-Pi relay board

 

If you are familiar with ASCOM, these devices should all be familiar.

 

I can control the system from anywhere in my house using the GUI drivers

 

Attached is a screen shot running several applications.

I normally run 3 scopes, with 3 nitecrawler focusers and 3 ZWO cameras, each scope has its own R-Pi, running the camera, filterwheel, focuser, and rotator.

 

In this screen shot, there are controllers for 2 scopes.  The 2 purple windows are for one scope, the 2 gold windows are for the 2nd scope.  It does sometimes get a bit busy but you would expect that trying to run 3 scopes at the same time.

 

If anyone would like to work with this, let me know.  It is a work in progress put it is very far along.

 

 

Mark

Edited by mark77, 07 June 2020 - 01:20 PM.

[Amzoltai]

Impressive !

[tkottary]

So , are these drivers or individual clients?  Not a single unified imaging client ?

[mark77]

So , are these drivers or individual clients?  Not a single unified imaging client ?

Correct

 

There is a driver running on each R-Pi.  When it launches, it looks to see if there are any cameras, filter wheels, focusers, or rotators.  It creates an instance (C++ object) for each one it finds.  

 

Alpaca includes a discover protocol.  The driver responds to that discover protocol with what it has running.

 

My client applications, which currently include SWITCH, FOCUSER, DOME, and CAMERA send out a query using the discover protocol, then for each device it finds, it opens up a window for that device.  

 

It is possible to have more than one client app talking to a driver.  The driver behaves kind of like a web server, in that it can have multiple clients talking to it.  Of course, you dont want 2 different apps telling the camera to do different things.  

 

This does bring up questions about security, but the discovery protocol only works on the local subnet so an app outside of the subnet cant find them.  However, if you know the IP address and port, they can be accessed from anywhere.

 

 

The Alpaca protocol does NOT define how the imaging app has to be set up.  You could have one app that talks to all of the devices and would be a "single unified imaging client"

 

For example, my "Camera" client talks to the camera and the filter wheel.  My focuser client tqlks to the focuser and to the rotator.  This is kind of non-transparrent since the MoonLite NiteCrawler focuser has the rotator built in.  However, from an implementation stand point, I have them show up as separte Alpaca devices to make sure I follow the Alpaca rules and can pass the Alpaca test proceedure.

 

On the camera interface, Alpaca/ASCOM define the ability to tell the camera to take a picture and then to retrieve the picture.  I have this mostly implemented but my client app does not use it.  I have the R-Pi store the image locally in both FITS and JPEG format,  much faster.  I have the R-Pi mounted as a file server on my desktop and I can access the image files directly.

[Bob Denny]

Mark -- BooYah!! Thanks for all you've done! People will benefit greatly from your work. 

 

One important correction:

 

Alpaca is a new protocol that was released at NEAF 2019.  It was designed by Bob Denny, the same guy that designed the original ASCOM prtocol.

I didn't design Alpaca (though I came up with the name and logo ha ha).. I have been the spokesperson for ASCOM Alpaca and a core team member. The design of the REST protocols, and the subsequent work that has gone into integrating the discovery capability and auto-link from COM to Alpaca that are part of the new ASCOM Platform 6.5 (release this coming week), and the developer tools you'll see rolling out in the near future, are the work of Peter Simpson and Daniel Van Noord. Peter originated the REST implementation and has continued to build out the infrastructure that allows interoperation between WIndows apps and COM or Alpaca drivers without needing any app changes at all. Daniel designed the discovery protocol and implemented it in a bunch of languages to prove it out, and is working on a large set of cross-platform developer tools. We owe them a huge thanks!

Edited by Bob Denny, 07 June 2020 - 04:50 PM.

[mark77]

Bob

 

Thanks for the correction, is it correct then to say the "ASCOM team" designed Alpaca.

 

I have to finish the image transmission in the camera driver so that I can get my camera code to pass CONFORM.  Then I should be close to release of version 1.  I dont want to make a public release until I pass all of the CONFORM tests.  I would however, work with individuals before that.

 

I also need to finish the paper I started writing about all of the additions/embellishments I have added.

 

 

Mark

[mikefulb]

Are you writing drivers from the ground up or leveraging something like INDI?

[Iver]

Hi Mark, I'm having a difficult time understanding what you are doing so please forgive my ignorance of INDI and ALPACA. I have always worked with Windows and ASCOM. It looks like you added support for at least 3 cameras that have both INDI drivers and ASCOM drivers. How does what you are doing fit in with ALPACA?

[mark77]

mikefulb and Iver

 

Very good questions....

 

Indi and ASCOM/Alpaca are 2 separate protocols that do not interact. In one way you could look at them as competitors.  I was going to go down the Indi path until Bob Denny introduced Alpaca at NEAF 2019 (April).  He convinced me that Alpaca was more in line with what I wanted to accomplish.

My personal opinion is that Alpaca is much easier to understand and implement than Indi.

 

 

Alpaca uses TCP/IP with http get and put commands that receive a JSON packet as a response.

 

I have written everything from the ground up.  I am using normal linux sockets library, built into the linux system.  I then do everything, I even wrote my own JSON parser.  I did not like any of the ones I found.

 

So for you software types out there, I have a base class that implements all of the TCP stuff and parsing, then sub classes to implement each driver, and sub classes from there for each camera type or focuser type.  The class structure allows for easy addition of new specific drivers.  For example, I already have 5 cameras implemented (not all of them complete yet).  The base class implements everything that is common among the drivers such as the common commands and the discovery protocol.

 

For the applications, again, I have a base class that implements the GUI and sub classes for each of the controller types, i.e. camera, focuser, switch, dome.

 

I wrote my own GUI from the ground up as well.  I use openCV for my graphics and implemented all the rest myself.  I have been doing GUI programming for 35 years.  (GUI = Graphical User Interface)

 

It is important to note that there is ZERO requirement to use my controller apps.  My Alpaca drivers are written to the Alpaca spec.  Anything that speaks Alpaca can talk to them, including any existing ASCOM program via the ASCOM/Alpaca bridge that you can install on your computer running an ASCOM application.

 

Similarly, my controller apps should talk to any other Alpaca driver written by someone else.

 

So, if you are running an ASCOM program on your PC to control things and want to put a Raspberry Pi to control a camera, focuser, rotator, or filter wheel on the scope.  My driver will allow you to do that and integrate into your EXISTING ASCOM application with no updates or changes in the app.  All you have to do is install and configure the ASCOM/Alpaca bridge.

 

Please, keep the questions coming

 

Mark

[TelescopeGreg]

This has been debated before, so I guess the question is, what's changed?

 

I run Astroberry on a Raspberry Pi 4B at my mount.  Everything's running locally, with VNC from a laptop or tablet being used for the screen and keyboard/mouse.  CCDciel, ASTAP, PHD2 on mine, but many use KStars / EKOS.  INDI underpins everything, and it all works just fine.  If I wanted to, I could split things into client / server between the laptop and Pi, using the same INDI underpinnings and with no change to the user experience.  The various equipment settings are easily found and managed.

 

Why change?

[mark77]

Greg

 

No reason to change at all, "If it ain't broke, dont fix it"

 

KStars utilizing Indi protocol is a good system.  I use KStars as my telescope planetarium program because the my telescope controller is open source Raspberry Pi TSC (Two Stepper Control) and it speaks LX200 which is compatible with KStars.

 

I use Alpaca and my own code for everything else. I hope to eventually add Alpaca support to TSC.

 

The only real advantage that Alpaca has over Indi is integration with existing ASCOM systems. There are also some fundamental design differences.

 

As I stated above, it is my opinion that from a software implementation stand point, Alpaca is easier to understand and implement than Indi.

 

Mark

[mikefulb]

Mark,

 

 Thanks for your answer.   I probably wasn't clear in my question.  Are you writing the hardware level from scratch - for example using the ASI SDK to talk to cameras, writing code to send/receive the Moonlite serial protocol, etc? 

 

 I was just thinking it could be interesting to leverage all the hard work that has gone into hardware support  in indilib.   Unfortunately I don't think there is an easy way to use that code other than via talking to an indiserver but it is not that heavy.

 

 I suppose if licenses are compatible you could lift a lot of code from the indi sources to get a head start.  I haven't investigated this so it might not be an option.

 

 I have written ASCOM, INDI and Alpaca drivers and I am finding I like using the Alpaca model for my simple projects.  I find it is trivial to throw together a quick test Alpaca driver in python.  Whereas having to use visual studio/windows for ASCOM is a deal killer for me and indilib involves C++ which I'd rather avoid for the simple things I'm working on.

[mark77]

Mark,

 

 Thanks for your answer.   I probably wasn't clear in my question.  Are you writing the hardware level from scratch - for example using the ASI SDK to talk to cameras, writing code to send/receive the Moonlite serial protocol, etc? 

 

 I was just thinking it could be interesting to leverage all the hard work that has gone into hardware support  in indilib.   Unfortunately I don't think there is an easy way to use that code other than via talking to an indiserver but it is not that heavy.

 

 I suppose if licenses are compatible you could lift a lot of code from the indi sources to get a head start.  I haven't investigated this so it might not be an option.

 

 I have written ASCOM, INDI and Alpaca drivers and I am finding I like using the Alpaca model for my simple projects.  I find it is trivial to throw together a quick test Alpaca driver in python.  Whereas having to use visual studio/windows for ASCOM is a deal killer for me and indilib involves C++ which I'd rather avoid for the simple things I'm working on.

Mike

 

Yes, I am writing at the ASI SDK level and I talk to the moonlite directly via USB.  I have full support from Moonlite. My focuser app will also go direct to the focuser and bypass Alpaca all together if so desired.

 

I have been selecting cameras and other devices strictly on the availability of linux drivers for those devices and more specifically if they have drivers for ARM-32 bit (Raspberry Pi).

 

I have found that trying to adapt other code is often more difficult and takes longer than writing things from scratch.  This is especially try for the camera drivers.  I have written the base class to implement the Alpaca/ASCOM command/data model.  Trying to adapt some other driver to fit into that would be difficult.  It is easier to write new code to fit the frame work.

 

I have been programming since 1977 and have been programming in C since 1985.  I do not know python and I am very comfortable with C/C++.  I do hope to learn Python sometime soon.

 

I tried to start writing INDI drivers before I heard about Alpaca and could not find good documentation on the architecture.  If you have insight to how to write INDI, I would be interested in seeing that. 

 

Mark

[Iver]

       Hi Mark, thanks for your response. If I were to install one of your camera drivers on a Pi and then install the ASCOM/Alpaca bridge does the camera then connect through the ASCOM camera chooser in my imaging software?  Is there a way to control and Indi camera on a Pi from an ASCOM app?

[mark77]

       Hi Mark, thanks for your response. If I were to install one of your camera drivers on a Pi and then install the ASCOM/Alpaca bridge does the camera then connect through the ASCOM camera chooser in my imaging software?  Is there a way to control and Indi camera on a Pi from an ASCOM app?

Iver

 

This is correct except for the Indi part.  Leave Indi out of it.  Your ASCOM app would talk to ASCOM devices.  The ASCOM/Alpaca bridge just changes the transport mechanism from Windows COM to TCP/IP JSON (a.k.a Alpaca).  The HTTP GET/PUT command is sent from the bridge to the Alpaca device (my R-Pi Driver).  A JSON message is returned with the results from that command.  The bridge converts it back to a Windows COM message and returns it to your app.

 

There is an exact one-to-one mapping of ASCOM messages to ALPACA messages.  This is intentional.

 

For my camera driver, I have to finish testing the image data return.  Once I do that (working on it this week).  It will be ready for release.

 

Mark

 

 

p.s.  No, there is not a way to control an Indi camera from an ASCOM program, at least not to my knowledge. I suppose it could be possible to write a translator app but it would probably be tricky at best.

Edited by mark77, 09 June 2020 - 07:51 AM.

[asl547]

Mark, I may be a little bit confused by the terminology. At the beginning of the thread you described your camera “client,” which I understand from your description to be an Alpaca client that can discover and control one or more active Alpaca camera drivers (a number of which you also have written). In your most recent post you refer to planned release of your camera “driver.” The Alpaca architecture description on the Ascom website shows Alpaca clients that can control Alpaca drivers, and Alpaca drivers that can be controlled either by Alpaca clients, or by Windows-Ascom clients through an Ascom to Alpaca “remote client drivers” interface. Can your Alpaca camera drivers (running, for example, on a RPi) be controlled by a separate Windows computer that is running an Ascom client (such as TheSkyX Windows version or Sharpcap), as distinguished from being controlled by your “CAMERA” application [Alpaca client, I think](presumably running on a Linux computer)? From your initial post and the general idea behind Alpaca, I assume that the answer is “yes.” If so, will you be releasing the camera “drivers,” for testing with Windows-Ascom clients as well as your Linux “CAMERA” application (client)? [Ultimately, I may be interested in having camera(s), filter wheel, and focuser connected to an on mount RPi running Alpaca drivers, controlled remotely by TheSkyX and/or other programs running on a separate Windows or Linux laptop.]

Edited by asl547, 09 June 2020 - 08:22 AM.

[mark77]

asl547

 

 

You got it exactly correct.....

 

I have to finish testing the image transfer from my Alpaca camera driver and then it should be ready for beta testing.  There is an ASCOM test tool called CONFORM.  Once I can get my camera driver to pass that test, then I will be ready to release it.  Until it passes CONFORM, it simply wont work 100%.

 

To be clear (and to repeat what you said above):

 

I have Alpaca drivers running on Linux (primarily R-Pi, but will run on Ubunto as well).

These drivers will work with any program that speaks Alpaca.

AND...

These drivers will be able to be used with any existing ASCOM compliant software program running on Windows, provided you have the ASCOM/ALPACA bridge installed.

 

I also have written several "client" apps that run on Linux that talk to the Alpaca drivers. These client apps can run on a different computer OR they can run on the same computer as the driver is running.

 

Mark

[Iver]

OK, so your plan is to make it open source so that others could write drivers?

 

Thanks for the info!

[mark77]

OK, so your plan is to make it open source so that others could write drivers?

 

Thanks for the info!

Iver

 

Yes I am going to make it open source.  I dont want to do a public release until I get the CONFORM testing fully passed.  Until then I will be glad to send the source to individuals if they contact me directly.

 

I probably should write some developer docs as well.

 

Mark

[mikefulb]

Great stuff Mark you've got my interest please keep giving us updates.

[glancey]

Iver

 

Yes I am going to make it open source.  I dont want to do a public release until I get the CONFORM testing fully passed.  Until then I will be glad to send the source to individuals if they contact me directly.

 

I probably should write some developer docs as well.

 

Mark

Can you just upload your code to github?

[mark77]

Can you just upload your code to github?

I will eventually.  I dont want to do a public release unfinished code.  I want to get everything passing the ASCOM/Alpaca "CONFORM" test before I make a full public release.

 

Right now, the camera driver works fine with my client application but it will not work with other ASCOM clients so in that regard it is not finished/broken.

 

Getting this finished is on my to-do list for next week.  Most of my other drivers are passing so I am doing pretty well.

 

I also need to work on some documentation.

 

Mark

[tjay]

I'm super interested in seeing the progress on this. Are you able to handle high frame rate video?

[mark77]

I'm super interested in seeing the progress on this. Are you able to handle high frame rate video?

I am currently working very hard on video on this project.  With a ZWO ASI120MC-S (USB3.0) saving as MPEG4, I am running 12 to 20 fps.  I can get higher speed in mono without compression.  I will be sure to keep some accurate numbers for various combinations and report back on it.

 

I do images and video in a way that is NOT part of the Alpaca spec in that I save them locally on the R-Pi.  I feel that trying to transfer image data in the Alpaca mode for video frame rates is not practical.

 

Bottom line is that I am working on it very hard and well try to get the best performance I can out of it.

 

I have to try various un-compressed formats to see if I can get higher frame rates.

 

 

Mark

[Iver]

Hi Mark, I hope you will remember to post a github link here when your project passes "CONFORM", and

 if someone with an SBIG camera that can code might consider writing an ALPACA driver for us ASCOM driver less SBIG users!