Table of Contents

  • TOC

Key Takeaways

  • Possible future job opportunity as Research Software Engineer (RSE)
  • Journal of Open Source Software (JOSS) as publication venue for software artifacts
  • Possible source of funding: Catalyst Fund
  • Discovered Code for Society as governance for ENTS
  • The Turning Way is a good resource for reproducible data science
  • UC OSPO Office Hours for information

Opening Session

Opening Remarks: Chancellor Cynthia Larive, UC Santa Cruz

  • Focus on advancing research and innovation in open source
  • Human genome project good example, beating out industry and keeping it closed source
    • Genome browser gets 1 mil access per day
  • New term “Open Science”
  • UCSC leads open source across the UC system
  • Contributor Catalyst: Allows students access to contributing to projects

Context Setting: James David and Carlos Maltzahn (CROSS)

  • Discussion focused on prevent degradation of research software
  • AI has an impact on open source
  • CROSS was created for Ceph (clustered Linux storage)
    • Sage Wile did the original work and founded DreamHost
  • CROSS partially created/funded through Sage
    • Initial backers: Toshiba, Micron, one other
    • Got up to 8 members
    • Experience led to the OSPO offices
  • Difficult to maintain software after graduation because of funding

Panel 1: Open Source Ecosystems as a Pathway to University Innovation

Moderator: Emily Lovell; Panelists: Daniel Fremont, Leilani Gilpin, Katia Obraczka, J. Xavier Prochaska

Project Summaries

Daniel Fremont (CSE): Scenic

  • probabilistic programming language
  • Use to model cyber physical systems
  • Have ML implemented
  • Safety critical system
  • Allow for the definition of an environment.
  • Funded by POSE
  • Trying to create a way to maintain project beyond POSE
  • Create 501(c) foundation

Lellani Gillpin (CSE)

  • Trying to make AI explainable and interpretable
  • Wrappers and monitors to focus on outputs from models
  • Collaborate with Daniel
  • TouchingSpace: CV for the visually impaired
  • Educational components focusing on AI.
    • Open source foundation for educational tools
    • Building in code review to the educational system
  • (Emily) companies are asking students to review AI generated code

Katia Obraczka (CSE)

  • Network Simulation Bridge (NSB)
  • Tool to assist their own research in networking
  • Funded by POSE
  • NSB is a narrower tool, looking at integration into existing open source foundation

J. Xavier Prochaska (Astronomy and Astrophysics)

  • Open source is multidisciplinary
  • PypeIt, astronomy processing packaging
  • Used in major observatories internationally
  • Not large quantity of users, but very useful to a few with funding
  • Uses GenAI for code and documentation

Questions

Why was open source chosen for your project?

  • (Daniel) Wanted something people to use and extensible. Now more company interest because open source is transparent.
  • (Katia) Had interest from other collaborators. Open source as a tool for collaboration. Additional educational benefits.
  • (Xavier) Focused on community.

How does the university sees the benefit of OSS?

  • (Katia) University is “open science”. Open source allows for longevity. Open source allows for better collaboration externally with regards to IP.
  • (Leilani) Papers are no longer viable for AI research. Tools can have higher impact.
  • (Daniel) In academia, open source count as impact such as papers.

What are resources university has provided?

  • (Xavier) Observatory initially provided funding.
  • (Katia) Not a good experience with the tech transfer office. Don’t seem to understand the benefits of open source software. Berkley has historically been good for open source software.
  • (Daniel) Corporate world likes external entity to work with rather than the university. OSPO was helpful in the “foundation” aspect of open source.

How have you navigated sustainability challenges?

  • (Daniel) POSE program was good forcing function for sustainability. Workshops were helpful. Different models ie. fiscal sponsorship where an entity specializes in legal, financing, etc.
    • Examples are: code for science and society, Node something??, and more
  • (Xavier) Distributed everywhere, but no set home for their software
  • (Katia) Possibility for different open source projects, can share engineering resources. Genomics institute maintains a group for system engineers.

What is the impact of AI on open source projects?

  • (Leilani) old school and has not seen good experience with AI usage.
  • (Xavier) Still requires human code review. AI lacks the skills to project and software manage the project. Want a human interface at both ends of the project.
  • (Daniel) Knowledgable users can accelerate the work. Can reduce the work needed to maintain the project. AI can integrate with existing project structure (tests, CI). Has major concerns about education with AI cutting out the first level of software engineering.

(Audience) What happens when the project outlives the research?

  • (Daniel) Remove the burden of maintenance outside of the research group.
  • (Emily) Always an option for a project to be open source.

(Audience) Is it reasonable for a person to just maintain a project?

  • (Katia) Keep evolving the tool such that new features are developed, not just maintain the tool.
  • (Carlos) OSPO has goal to recruit new students that can help support projects.

(Audience) Successes/challenges for developing an ecosystem

  • (Xavier) Sent the money to the community such that the project could be maintained by the people using it.
  • (Daniel) POSE allow for simulator testing for the infrastructure. Adding features would open up new use cases. Didn’t get much out of the interviews. Trying to ask: how much money to ask for from companies.

(Audience) Was the POSE interviews useful?

  • (Katia) found the process extremely useful. Found out that their tool was not useful for everyone.

Scenic TouchingSpace NSB PypeIt

Keynote - Research software engineering and research software engineers: Successes and challenges.

Daniel S. Katz, University of Illinois Urbana-Champaign

Link to slides

  • Community of Research Software Community (RSE)
    • Research don’t know good software practices
    • Software engineers don’t understand research context
    • RSE fits between these two extremes
  • Collaborations workshop 2012 realized there was No common title, hence RSE
    • Now there is a society of RSE
  • US-RSE (https://us-rse.org)
    • Daniel is a co-founder and serves on the board
  • Research Software Alliance (ReSA) extends beyond research to publishers, other software supporters
  • Journal of Open Source Software (JOSS)
    • Software equivalent to accepted paper
    • https://doi.org/10.31274/jlsc.18285
  • Aim to provide reward for human work
  • Publish papers with minimum effort (>3 months of work, >1000 LOC)
  • Got a lot of submissions just for the sake of publication
  • 75-80% rejection rate now
  • Research Software Engineering in the age of generative AI
    • Group discussion of the effects of AI
  • Catalyst Fund
    • Grants for people building/verifying, and teaching research software

Panel 2: Campus Infrastructure Supporting Open Source and Research Impact

Moderator: James Davis; Panelists: Yi Zhang, Jason Nielsen, Benedict Paten, Jeff Weekley

Background

Jason Nielson Director of the Santa Cruz Institute for Particle Physics (SCIPP).

Yi Zhang Lead of generative AI at UCSC and Silicon Valley extension. Organizes events to support community. Does NOT have computing resources.

Benedict Paten Associate director of human genome project. Does all things genomics. Struggle to run a compute cluster and hire two full time system engineers. Funded federally and through private donations.

Questions

How do maintain funding for existing funding?

  • (Benedict) Funded through NHS for maintain databases. Use limited UCSC funds as well.
  • (Yi) More ad-hoc support. Done on a case by case basic for the needs of a project. Some donated/bought hardware.
  • (Jason) Leverage nationalized infrastructure for existing projects. Difficult to get compute infrastructure for new projects or “trail runs”. Use institutions funds for perspective projects.

Why should the university care about providing resources?

  • (Jason) Providing infrastructure is a pathway for innovation.
  • (Yi) Allow for better tech transfer to a company
  • (Benedict) New faculty hires have obligations to physical spaces, but there’s no discussion on compute infrastructure. University will distinguish themselves among compute infrastructure available.

On-site or external compute? Limited by researchers?

  • (Benedict) Need DevOps skills for maintaining infrastructure. Cloud more expensive.
  • (Jason) Biggest gains is from RSEs to get the most out of the hardware. Lacking case studies on how much RSE impact cost.
  • (Yi) Should be the role of the IT department for maintaining compute.

How can you trust AI research?

  • (Benedict) Concerns about checking gen AI code. Major applications in medical where peoples lives may be at risk.
  • (Yi) AI can not fill the gap between the idea in someone’s head vs the result from the AI

Keynote: Everything old is new again: the deepened importance of humans learning together in the age of AI

Kirstie Whitaker, Berkeley Institute for Data Science (BIDS)

Slides

  • Working reproducibility benefits you in the future
  • Publication requires new research, no emphasis placed on reproducing results
  • Lightcone as a AI tool for reproducible research
    • Able to view decisions made for the paper, assumptions, constants
    • Shift from authorship about writing the code TO advancement of knoweldge
  • BIDS annual report as a good reference for current state
    • Good reference for what is important for Berkley
  • Hiring for Lightcone Research and BioJupyter

Directed Discussion: Institutional Infrastructure, Research & OSPOs

Moderators: Stephanie Lieggi and Hari Kuttivelil

List of open source offices in attended.

UCSF

  • No CS department
  • Made to coordinate computation resources between different labs

Berkley

  • Part of BIDS which is a part of computer science
  • Maintain scientific python tools

Georgia Tech

  • Under research computing and library
  • Establishing presence
  • Provide funding support for researchers
  • Hosting of large datasets
  • Funded with Sloan?? foundation

Syracuse University

  • Fund students to work on open source projects
  • Funded with Sloan?? foundation
  • Created course for open source contributions
  • Hosted workshops for faculty

University of Washington Madison

  • Sits inside of the data science institute
  • Standard way to providing information to people?
  • Heavily involved in high performance computing
  • Internship program to support students

University of Illinois Urbana-Champaign

  • No OSPO, but has RSE group
  • Support research activities on campus through students
  • Have local compute cluster

Closing Remarks

Roberto Manduchi, Baskin Engineering Interim Dean