Jump to content

  •  

CNers have asked about a donation box for Cloudy Nights over the years, so here you go. Donation is not required by any means, so please enjoy your stay.

Photo

Image Repository

  • Please log in to reply
11 replies to this topic

#1 ispybadguys

ispybadguys

    Viking 1

  • -----
  • topic starter
  • Posts: 536
  • Joined: 31 Jan 2021
  • Loc: Albuquerque,NM

Posted 03 March 2024 - 05:57 PM

We are thinking of setting up an image repository for our astronomy club where members can share calibrated sub-frames with the idea of doing collaborative collections. Does anybody know of any similar efforts?

 

Thanks

 

Kurt


  • Palmito likes this

#2 TOMDEY

TOMDEY

    James Webb Space Telescope

  • *****
  • Posts: 15,764
  • Joined: 10 Feb 2014
  • Loc: Springwater, NY

Posted 03 March 2024 - 06:14 PM

Good in theory, can get complicated in practice. We did that at work (Kodak) --- and even there, with all the paid professionals and support crew --- process management was frequently a nightmare. The responsibilities, formats, management  --- and (especially) disagreements on how to handle all that. Such an undertaking (at a club) requires someone enthusiastic and knowledgeable to step forward and offer to coordinate and champion the effort. Looks like you are the volunteer?   Tom


  • Palmito and ispybadguys like this

#3 Palmito

Palmito

    Vostok 1

  • -----
  • Vendors
  • Posts: 181
  • Joined: 16 Nov 2018
  • Loc: Lausanne, Switzerland

Posted 04 March 2024 - 08:52 AM

For our group projects (50 in the group, up to 22 contributors on last project), we use a NAS.
I create a share link, and I create a folder with filters sub folders for each contributor.

 

But there are many other possibilities, the NAS is just convenient, a FTP server is another one.

Google drive or one drive works too, but would require each contributor to pay for sufficient storage capacity, which makes it unpractical.

 

We will come up with tutorials covering all aspects of collaborations in about 1 to 2 months.

You can follow the link to our website from here : https://www.cloudyni...quest-for-oiii/

 

Carl


  • ispybadguys likes this

#4 ispybadguys

ispybadguys

    Viking 1

  • -----
  • topic starter
  • Posts: 536
  • Joined: 31 Jan 2021
  • Loc: Albuquerque,NM

Posted 04 March 2024 - 09:15 AM

Carl

 

Thanks for the info. I looked as a couple of options, Our own NAS and some kind of cloud storage. Cloud storage for sub-frames is very expensive. If we have 40TB of cloud it costs about $1,000/month plus about $3 for an average target set download. It seems that Sharepoint might be the way to go. It is $5/person/month and each person gets 1TB. That seems like an option.

 

Kurt


  • Palmito likes this

#5 Palmito

Palmito

    Vostok 1

  • -----
  • Vendors
  • Posts: 181
  • Joined: 16 Nov 2018
  • Loc: Lausanne, Switzerland

Posted 04 March 2024 - 09:25 AM

Hi Kurt,

 

Yes cloud storage can be super expensive, advantages are to be able to rely on their backups, security and the connection speed that will be high worldwide.

5$ a month per person seems reasonable yes.

 

On my NAS (I am in Europe and I have a 1gigabit download/upload internet connection), sometimes it is super slow in the US.

But it does work sufficiently for us, to give you an idea we use approx 0.5TB of submitted files per project.

 

Carl



#6 GaryShaw

GaryShaw

    Gemini

  • *****
  • Posts: 3,048
  • Joined: 28 Nov 2017
  • Loc: Massachusetts / Wyoming

Posted 04 March 2024 - 09:42 AM

Hi Kurt

Check out these guys. There’s also a substantial post by them here in CN today.

Gary

 

https://deepskycollective.com/home


  • Palmito likes this

#7 Palmito

Palmito

    Vostok 1

  • -----
  • Vendors
  • Posts: 181
  • Joined: 16 Nov 2018
  • Loc: Lausanne, Switzerland

Posted 04 March 2024 - 09:52 AM

Hi Kurt

Check out these guys. There’s also a substantial post by them here in CN today.

Gary

 

https://deepskycollective.com/home

Hi Gary,

Yes I posted it :-)

Carl


  • ispybadguys likes this

#8 choward94002

choward94002

    Surveyor 1

  • *****
  • Posts: 1,649
  • Joined: 25 Jul 2016
  • Loc: Central AZ

Posted 04 March 2024 - 11:05 AM

We are thinking of setting up an image repository for our astronomy club where members can share calibrated sub-frames with the idea of doing collaborative collections. Does anybody know of any similar efforts?

 

Thanks

 

Kurt

There used to be an old joke ... "What is the fastest, cheapest way to send 10TB across the country?" ...and the answer was "FedEx"

 

I did a collaboration with some folks a few months ago with M101 and the SN that happened there ... I pushed nearly 50TB of data to the 10 folks doing the processing over a two week period, doing that using anything other than 50 quad layer BluRay discs would have been both costly and silly ...

 

When I supply data to folks these days I always do it on discs ... someone asks for a target, I pull the data from cold storage (or warm from the NAS), burn it onto a quad layer, toss it into a mailer and everyone is happy ... they have data that will never get lost or degrade for about $15 including shipping


  • Palmito likes this

#9 Palmito

Palmito

    Vostok 1

  • -----
  • Vendors
  • Posts: 181
  • Joined: 16 Nov 2018
  • Loc: Lausanne, Switzerland

Posted 04 March 2024 - 11:38 AM

There used to be an old joke ... "What is the fastest, cheapest way to send 10TB across the country?" ...and the answer was "FedEx"

 

I did a collaboration with some folks a few months ago with M101 and the SN that happened there ... I pushed nearly 50TB of data to the 10 folks doing the processing over a two week period, doing that using anything other than 50 quad layer BluRay discs would have been both costly and silly ...

 

When I supply data to folks these days I always do it on discs ... someone asks for a target, I pull the data from cold storage (or warm from the NAS), burn it onto a quad layer, toss it into a mailer and everyone is happy ... they have data that will never get lost or degrade for about $15 including shipping

Haha, reminded me of a story, in South Africa they tested sending a 4GB file through ADSL vs a pigeon.

Long story short the pigeon won, took him ~2 hours for 37mi, ADSL transfer was only at 4% :-D

https://en.wikipedia..._Avian_Carriers



#10 ispybadguys

ispybadguys

    Viking 1

  • -----
  • topic starter
  • Posts: 536
  • Joined: 31 Jan 2021
  • Loc: Albuquerque,NM

Posted 04 March 2024 - 04:36 PM

For our group projects (50 in the group, up to 22 contributors on last project), we use a NAS.
I create a share link, and I create a folder with filters sub folders for each contributor.

 

But there are many other possibilities, the NAS is just convenient, a FTP server is another one.

Google drive or one drive works too, but would require each contributor to pay for sufficient storage capacity, which makes it unpractical.

 

We will come up with tutorials covering all aspects of collaborations in about 1 to 2 months.

You can follow the link to our website from here : https://www.cloudyni...quest-for-oiii/

 

Carl

Carl

 

If we want to contribute to or access the data is this possible?

 

Kurt



#11 auroraTDunn

auroraTDunn

    Viking 1

  • -----
  • Posts: 650
  • Joined: 01 Sep 2017
  • Loc: North of Boulder, Co.

Posted 04 March 2024 - 06:37 PM

You could always go the route I do when working on, shall we say non-co-funded support, with outside universities and research institutions (I exist only to help others so 8 hours of funded help leaves 16 for non-funded -shhhh) using my own memory tower, making the tower its own server. Not the cheapest upfront, pending how big you want to go, but then you get ultimate flexibility. I'm highly prone to using G-Technologies (https://support-en.g-technology.com/) because I can plug and play different drives making them archivable, remove and plug a new one in. Plus I'm used to using them, back in the day, when I worked as a CG/VFX TD for movies and TV shows. I just keep a small system setup to make them work as a cloud server.



#12 Palmito

Palmito

    Vostok 1

  • -----
  • Vendors
  • Posts: 181
  • Joined: 16 Nov 2018
  • Loc: Lausanne, Switzerland

Posted 05 March 2024 - 03:54 AM

Carl

 

If we want to contribute to or access the data is this possible?

 

Kurt

 

Kurt,

Alas we are at full capacity at the moment, there are some ongoing discussions to share stacks with the scientific community, but it is not in place yet I am afraid.

 

Carl




CNers have asked about a donation box for Cloudy Nights over the years, so here you go. Donation is not required by any means, so please enjoy your stay.


Recent Topics






Cloudy Nights LLC
Cloudy Nights Sponsor: Astronomics